runlist event interrupt is not needed to be enabled as
s/w polls for preemption completion for preempts issued
in RUNLIST_PREEMPT. Even though it is not enabled, intr
will get set in fifo_intr_0 status register whenever
RUNLIST_PRREMPT is successfully completed. Since intr
is disabled, fifo intr will not be triggered but it will
be handled during handling of other fifo interrupts
whenever fifo intr is triggered.
Bug 2039371
Change-Id: I0817c2b6e9f3f14958ca7c738392bc67875be5d5
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1630283
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
During bringup and before nvlink is up GV100 on the DDPX platform operates
with a very, very slow sysmem link. In order to get sysmem test to pass
it is neccesary to significantly increase most timeouts by an order the
magnitude.
Bug 2040544
Change-Id: I26858afde4ae80c70f86b47cfff674b6b00b5bf8
Signed-off-by: David Nieto <dmartineznie@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1627417
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Context TSG teardown procedure:
1. Disable scheduling for the engine's runlist via PFIFO_SCHED_DISABLE.
This enables SW to determine whether a context has hung later in the
process: otherwise, ongoing work on the runlist may keep ENG_STATUS from
reaching a steady state.
2. Disable all channels in the TSG being torn down or submit a new runlist
that does not contain the TSG. This is to prevent the TSG from being
rescheduled once scheduling is reenabled in step 6.
3.
a)Initiate a preempt of the TSG by writing NV_PFIFO_PREEMPT
with the TSG's ID and the TYPE set to TSG if TSG id is known else
do 3b
b)Initiate a preempt of the engine by writing the bit associated with its
runlist to NV_PFIFO_RUNLIST_PREEMPT. This allows us to begin the preempt
process prior to doing the slow register reads needed to determine
whether the context has hit any interrupts or is hung. Do not poll
NV_PFIFO_RUNLIST_PREEMPT for the preempt to complete.
4. Check for preempt done
5. If a reset is needed as determined by step 4:
a. Halt the memory interface for the engine (as per the relevant engine
procedure).
b. Reset the engine via PMC_ENABLE.
c. Take the engine out of reset and reinit the engine (as per the relevant
engine procedure)
6. Re-enable scheduling for the engine's runlist via PFIFO_SCHED_ENABLE.
Bug 200277163
Change-Id: I1e945a2c6b9845f365d6952109f6803309aa2270
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1599841
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Commit 81868a187f updated barrier
usage to use the nvgpu wrappers and in doing so downgraded many
plain barriers {mb(), wmb(), rmb()} to the SMP versions of these
barriers.
The SMP version of the barriers in question are only issued
when running on an SMP machine. In most of the cases mentioned
above this is fine since the barriers are present to faciliate
proper ordering across CPUs. A single CPU is always coherent
with itself, so on a non-SMP case we don't need those barriers.
However, there are a few places where the barriers in use (GMMU
page table programming, IO accessors, userd) where the barrier
usage is for communicating and establishing ordering for the
GPU. We need these barriers for both SMP machines and non-SMP
machines. Therefor we must use the plain barrier versions.
Change-Id: I376129840b7dc64af8f3f23f88057e4e81360f89
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1599744
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Check for interrupts or hangs while waiting for the preempt to complete.
During pbdma/eng preempt done polling, any stalling interrupts relating
to the runlist must be detected and handled in order for the preemption
to complete.
When PBDMA fault or CE fault occurs, the PBDMA will save out
automatically. TSG related to the context in which the fault occurred
will not be scheduled again until the fault is handled.
In the case of some other issue requiring the engine to be reset, TSG
will need to be manually preempted.
In all cases, a PBDMA interrupt may occur prior to the PBDMA being able to
switch out. SW must handle these interrupts according to the relevant handling
procedure before the PBDMA preempt can complete.
Opt for eng reset instead of waiting for preemption to be finished when
there is any stall interrupt pending during engine context preempt completion.
Bug 200277163
Bug 1945121
Change-Id: Icaef79e3046d82987b8486d15cbfc8365aa26f2e
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1522914
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: David Martinez Nieto <dmartineznie@nvidia.com>
Tested-by: David Martinez Nieto <dmartineznie@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
After unbinding channel, following fields in
channel status needs to be cleared manually:
ccsr_channel_enable_clr_true
ccsr_channel_pbdma_faulted_reset
ccsr_channel_eng_faulted_reset
Unbinding channel expected to clear all other
channel status fields.
Bug 1972365
Change-Id: Ibfd84df2f41adc2eb437a026acde3f3d618d7758
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1594671
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Use abstract nvgpu_rwsem as TSG channel list lock instead of the Linux
specific rw_semaphore.
JIRA NVGPU-259
Change-Id: I5f6c918464315e3d140bea0c61a619c3712619c1
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1579934
GVS: Gerrit_Virtual_Submit
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
For sync-point read map, create read only map per vm
and share with all channels that are using same vm.
Now restrict rw map to single syncpoint shim memory range.
JIRA GPUT19X-2
Change-Id: Ibd0b82d1cdb8861e1dbb073b27da1f9c9ab1d2ab
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1514339
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reading gr_pri_fe_chip_def_info_r() during
gv11b_init_fifo_setup_hw on RTL platforms is giving
"0xbadf1201" error because fecs part of priv ring
is still in reset. This needs to be fixed after
identifying relevant engine that needs to be
brought out of reset. Until that time, use constant
value from hw definition(whose value is 64):
gr_pri_fe_chip_def_info_max_veid_count_init_v().
Bug 1983643
Change-Id: I66f2b6491c9d444c6f6919e76c72ec33a904bc90
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1568139
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Tested-by: Seema Khowala <seemaj@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Add new API gv11b_fifo_tsg_verify_status_faulted() and use that as
g->ops.fifo.tsg_verify_status_faulted operation for gv11b/gv100
This API will check if channel has ENG_FAULTED status set, if yes it will clear
CE method buffer in case saved out channel is same as faulted channel
We need to write 0 to method count to invalidate CE method buffer
Also set g->ops.fifo.tsg_verify_status_ctx_reload operation for gv11b/gv100
Bug 200327095
Change-Id: I9d2b0f13faf881b30680219bbcadfd4969c4dff6
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1560643
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Add TSG enable/disable operations for gv11b/gv100
To disable a TSG we continue to use gk20a_disable_tsg()
To enable a TSG add new API gv11b_fifo_enable_tsg() since TSG enable sequence is
different for Volta than previous versions
For Volta it is sufficient to loop over all the channels in TSG and enable them
sequentially
Bug 1739362
Change-Id: Id4b4684959204c6101ceda83487a41fbfcba8b5f
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1560642
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reorganize HAL initialization to remove inheritance and construct
the gpu_ops struct at compile time. This patch only covers the fifo
sub-module of the gpu_ops struct.
Perform HAL function assignments in hal_gxxxx.c through the
population of a chip-specific copy of gpu_ops.
Jira NVGPU-74
Change-Id: I7c81edfa785a4ecafef41aae7b82d6b1707d294e
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1522554
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
HUB reports following memory sub-system interrupts:
a) ACCESS_COUNTER_NOTIFY:
GET != PUT for access counter notify buffer
b) ACCESS_COUNTER_ERROR:
HUB received a NACK (BAR2 fault) when writing the
notify buffer out to memory
c) MMU_ECC_UNCORRECTED_ERROR_NOTIFY:
Uncorrected ECC error detected by HUB MMU
d) MMU_REPLAYABLE_FAULT_NOTIFY:
GET != PUT for replayable fault buffer
e) MMU_REPLAYABLE_FAULT_OVERFLOW:
Overflow when writing to the replayable fault buffer
f) MMU_NONREPLAYABLE_FAULT_NOTIFY:
GET != PUT for non-replayable fault buffer
g) MMU_NONREPLAYABLE_FAULT_OVERFLOW:
Overflow when writing to the non-replayable fault buffer
h) MMU_OTHER_FAULT_NOTIFY: All other fault notifications from MMU
This change is to :
-Detect other fault notify
-Copy fault info from fault snap register for other fault notify
interrupt
-Detect and handle nonreplay/replay fault notify and fault overflow
-Copy fault info from fault buffer for nonreplay/replay fault
-Print fault info
JIRA GPUT19X-7
JIRA GPUT19X-12
Change-Id: Ifa08a4ebcd119a7d81c2eae3f52dc825d1ce3898
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master/r/1493394
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
GVS: Gerrit_Virtual_Submit
Changes to enable 64 subcontexts: 1 SYNC + 63 ASYNC
Currently all subcontexts with in a tsg can have only
single address space.
Add support for NVGPU_TSG_IOCTL_BIND_CHANNEL_EX for
selecting subctx id by client.
Bug 1842197
Change-Id: Icf56a41303bd1ad7fc6f2a6fbc691bb7b4a01d22
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: https://git-master/r/1511145
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
GVS: Gerrit_Virtual_Submit
When Host receives a page fault signal from a CE, Host will immediately
set _ENG_FAULTED bit in the NV_PCCSR_CHANNEL register for the channel
and will trigger a preempt of the TSG/channel.
A channel will only be scheduled when _ENABLED=1, _ENG_FAULTED=0
and _PBDMA_FAULTED=0 in pccsr_channel reg for the channel.
If a TSG has a faulted channel, Host will not schedule the entire TSG
agin until all _FAULTED bits from channels in the TSG are cleared by SW.
This function will be required for ce page fault handling.
JIRA GPUT19X-46
JIRA GPUT19X-12
Change-Id: Ib58dff7aa24aa144e970f11b5261877dec03f3e6
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master/r/1509776
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
When CE hits a page fault it needs to save out methods, it will
save it out to a per runqueue per TSG method buffer. The method buffers
(one per TSG runqueue) are allocated in BAR2 space during TSG creation
All channels in a TSG that are mapped to the same runqueue will point
to the same buffer.
S/w will insert channel's method buffer pointer in the channel's
instance block entries NV_RAMIN_ENG_METHOD_BUFFER_ADDR_LO and
NV_RAMIN_ENG_METHOD_BUFFER_ADDR_HI. Method buffer in memory will
be 32B aligned.
Eng method buffer allocated per tsg will be de-allocated during
tsg_release.
JIRA GPUT19X-46
Change-Id: Ib480ae5840d9815d24fe2eadc169ac3102854cd0
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master/r/1509747
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
- add fifo_gv11b to store usermode_regs
- consider channel_base and use usermode_regs when ring channel doorbell
It'll make kickoff code re-usable for vgpu.
Jira VFND-3796
Change-Id: Ia6974ccac137f201ad8763a7d372de81d5cca56b
Signed-off-by: Richard Zhao <rizhao@nvidia.com>
Reviewed-on: https://git-master/r/1510457
Reviewed-by: Aingara Paramakuru <aparamakuru@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
hw_chid is a relative id for vgpu. For native it's same as hw id.
Renaming it to chid to avoid confusing.
Jira VFND-3796
Change-Id: Ie94c1a15e9e45fc823d85790ce6a69da53a685bf
Signed-off-by: Richard Zhao <rizhao@nvidia.com>
Reviewed-on: https://git-master/r/1509531
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Pbdma which encountered the ctxnotvalid interrupt will stall and
prevent the channel which was loaded at the time the interrupt fired
from being swapped out until the interrupt is cleared.
CTXNOTVALID pbdma interrupt indicates error conditions related
to the *_CTX_VALID fields for a channel. The following
conditions trigger the interrupt:
* CTX_VALID bit for the targeted engine is FALSE
* At channel start/resume, all preemptible eng have CTX_VALID FALSE but:
- CTX_RELOAD is set in CCSR_CHANNEL_STATUS,
- PBDMA_TARGET_SHOULD_SEND_HOST_TSG_EVENT is TRUE, or
- PBDMA_TARGET_NEEDS_HOST_TSG_EVENT is TRUE
JIRA GPUT19X-47
Change-Id: If65ce1fcdbaebd6b1d8313fdddf9e3e0fa51e885
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master/r/1329372
GVS: Gerrit_Virtual_Submit
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Preempt type should be set to tsg and id should be set to tsgid
in fifo_preempt_r(). Preempt type channel and id set to channel
id does not initiate preemption.
Bug 200289427
Bug 200292090
Bug 200289491
Change-Id: I2ae96c0b9ca8a88a8405f42775744f0879994887
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: http://git-master/r/1497877
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Remove use of linux specifix header files
<linux/nvhost.h> and <linux/nvhost_t194.h>
and use nvgpu specific header file
<nvgpu/nvhost_t19x.h> instead
This is needed to remove all Linux dependencies
from nvgpu driver
Replace all nvhost_*() calls by
nvgpu_nvhost_*() calls from new nvgpu library
Jira NVGPU-29
Change-Id: I32d59628ca5ab3ece80a10eb5aefa150b1da448b
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1494648
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
fb_timeout and pbdma_timeout values are already set by h/w to init
values. No need to reinitialize.
JIRA GPUT19X-22
Change-Id: If6f1111f58940d51e53f028b046c42fa852221ee
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: http://git-master/r/1493458
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
In t19x, host1x supports sync point through memory mapped
shim layer. So sync-point operations implemented through
semphore methods signaling to this sync-point shim layer.
Added relevant hal functions for this in fifo hal.
JIRA GPUT19X-2
Change-Id: Ia514637d046ba093f4e5afa6cbd06673672fd189
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: http://git-master/r/1258235
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
