Current sequence in gk20a_disable_channel() is
- disable channel in gk20a_channel_abort()
- adjust pending fence in gk20a_channel_abort()
- preempt channel
But this leads to scenarios where syncpoint has
min > max value
Hence to fix this, make sequence in gk20a_disable_channel()
- disable channel in gk20a_channel_abort()
- preempt channel in gk20a_channel_abort()
- adjust pending fence in gk20a_channel_abort()
If gk20a_channel_abort() is called from other API where
preemption is not needed, then use channel_preempt
flag and do not preempt channel in those cases
Bug 1683059
Change-Id: I4d46d4294cf8597ae5f05f79dfe1b95c4187f2e3
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/921290
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
Instead of depending on clock frame-work, use platform data
for ptimer source rate. Removed ptimerscaling10x platform
data, and use ptimer source frequency to calculate
ptimerscaling factor.
Reviewed-on: http://git-master/r/819030
(cherry picked from commit dd291334d54dab80cab7eb1656dffc48a59610b4)
Change-Id: I7638ce9875a6e440bbfc2ba2da0d0b094b2700ff
Signed-off-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: http://git-master/r/827300
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Add new IOCTL NVGPU_IOCTL_TSG_SET_PRIORITY to allow
setting timeslice for entire TSG
Return error from channel specific IOCTL_CHANNEL_SET_PRIORITY
if the channel is part of TSG
Separate out API gk20a_channel_get_timescale_from_timeslice()
to get timeslice_timeout and scale from timeslice period
Use this API to get timeslice_timeout and scale for TSG and
store it in tsg_gk20a structure
Then trigger runlist update so that new timeslice values
will be re-written to runlist for TSG
Bug 200146615
Change-Id: I555467d034f81b372b31372f0835d72b1c159508
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/824206
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
When we induce a fake MMU fault, we do not have pointer to a channel.
Use the tsg pointer instead. Also remove the error print in case we
do not have ch pointer.
Change-Id: I14fd75d2b743244915bf32fe39de76097ef5c42f
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/819034
In gk20a_fifo_handle_sched_error(), we currently cancel
the timeout on all the channels
But this could cause us to miss one of stuck channel
hence, instead of cancelling, restart the timeout of channel
on which it is already active
Bug 200133289
Change-Id: I40e7e0e5394911fc110ab6fde39592b885dfaf7d
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/816133
Reviewed-by: Ishan Mittal <imittal@nvidia.com>
Tested-by: Ishan Mittal <imittal@nvidia.com>
To handle any of the pbdma interrupt, we currently write zero
to pbdma_method0 and then clear the interrupt
But this is insufficient since we cannot use same intr clear
method for all the interrupts
Hence, add intr specific routines to handle those interrupts
NV_PPBDMA_INTR_0_PBENTRY:
- fix the pb_header to have a null opcode
- fix the pbdma_method to have a valid nop
NV_PPBDMA_INTR_0_METHOD:
- fix the pbdma_method to have a valid nop
NV_PPBDMA_INTR_0_DEVICE:
- fix the pb_header to have a null opcode
- go through all pbdma_method0/1/2/3
-- if they contain host s/w methods, replace those
methods with a valid NOP
Bug 200134238
Change-Id: I10c284a6cdc1441f9d437cea65aae00d3c33a8c8
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/814561
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
A SCHED error might cause multiple channels' watchdogs to trigger
simultaneously
Hence, to avoid this conflict cancel watchdog timeout on all
channels before recovering from SCHED errors
Also, define API gk20a_channel_timeout_stop_all_channels()
to cancel wdt timeout on all channels
Bug 200133289
Change-Id: I8324c397891f0a711327b77d0677cd6718af6d01
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/810959
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
bug 1603226
host based timeouts use ptimer for detecting
timeouts. on gk20a and gm20b ptimer runs 2.6x
slower. scale the fifo_eng_timeout to account
for this
Change-Id: Ie44718382953e36436ea47d6e89b9a225d5c2070
Signed-off-by: Vijayakumar <vsubbu@nvidia.com>
Reviewed-on: http://git-master/r/799983
(cherry picked from commit d1d837fd09ff0f035feff1757c67488404c23cc6)
Reviewed-on: http://git-master/r/808250
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
Implement per-channel watchdog/timer as per below rules :
- start the timer while submitting first job on channel or if
no timer is already running
- cancel the timer when job completes
- re-start the timer if there is any incomplete job left
in the channel's queue
- trigger appropriate recovery method as part of timeout
handling mechanism
Handle the timeout as per below :
- get timed out channel, and job data
- disable activity on all engines
- check if fence is really pending
- get information on failing engine
- if no engine is failing, just abort the channel
- if engine is failing, trigger the recovery
Also, add flag "ch_wdt_enabled" to enable/disable channel
watchdog mechanism. Watchdog can also be disabled using
global flag "timeouts_enabled"
Set the watchdog time to be 5s using macro
NVGPU_CHANNEL_WATCHDOG_DEFAULT_TIMEOUT_MS
Bug 200133289
Change-Id: I401cf14dd34a210bc429f31bd5216a361edf1237
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/797072
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
bug 1625901
1) disable ELPG before doing GR reset when runlist update times out
2) add mutex for GR reset to avoid multiple threads resetting GR
3) protect GR reset with FECS mutex so that no one else submits methods
Change-Id: I02993fd1eabe6875ab1c58a40a06e6c79fcdeeae
Signed-off-by: Vijayakumar <vsubbu@nvidia.com>
Reviewed-on: http://git-master/r/793643
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
In gk20a_fifo_handle_sched_error(), we currently have a sequence
to identify failing engine (stuck on context switch) and
corresponding failing channel with its type
Separate out this sequence in new API
gk20a_fifo_get_failing_engine_data() so that it can be
reused from else where too
Bug 200133289
Change-Id: I3cef395170cf8990c014c7505c798fd6f2e37921
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/797070
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
bug 200114561
1) when handling sched error, if CTXSW status reads switch
check FECS mailbox register to know whether next or current
channel caused error
2) Update recovery function to use ch id passed to it
3) Recovery function now passes mmu_engine_id to mmu fault
handler instead of fifo_engine_id
Change-Id: I3576cc4a90408b2f76b2c42cce19c27344531b1c
Signed-off-by: Vijayakumar <vsubbu@nvidia.com>
Reviewed-on: http://git-master/r/763538
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Sachin Nikam <snikam@nvidia.com>
Add reference counting for channels, and wait for reference count to
get to 0 in gk20a_channel_free() before actually freeing the channel.
Also, change free channel tracking a bit by employing a list of free
channels, which simplifies the procedure of finding available channels
with reference counting.
Each use of a channel must have a reference taken before use or held
by the caller. Taking a reference of a wild channel pointer may fail, if
the channel is either not opened or in a process of being closed. Also,
add safeguards for protecting accidental use of closed channels,
specifically, by setting ch->g = NULL in channel free. This will make it
obvious if freed channel is attempted to be used.
The last user of a channel might be the deferred interrupt handler,
so wait for deferred interrupts to be processed twice in the channel
free procedure: once for providing last notifications to the channel
and once to make sure there are no stale pointers left after referencing
to the channel has been denied.
Finally, fix some races in channel and TSG force reset IOCTL path,
by pausing the channel scheduler in gk20a_fifo_recover_ch() and
gk20a_fifo_recover_tsg(), while the affected engines have been identified,
the appropriate MMU faults triggered, and the MMU faults handled. In this
case, make sure that the MMU fault does not attempt to query the hardware
about the failing channel or TSG ids. This should make channel recovery
more safe also in the regular (i.e., not in the interrupt handler) context.
Bug 1530226
Bug 1597493
Bug 1625901
Bug 200076344
Bug 200071810
Change-Id: Ib274876908e18219c64ea41e50ca443df81d957b
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Signed-off-by: Sami Kiminki <skiminki@nvidia.com>
Reviewed-on: http://git-master/r/448463
(cherry picked from commit 3f03aeae64ef2af4829e06f5f63062e8ebd21353)
Reviewed-on: http://git-master/r/755147
Reviewed-by: Automatic_Commit_Validation_User
Improve the error path and return values from function
gk20a_fifo_handle_sched_error()
- return true, when we trigger the recovery in this function
- otherwise, return false
No need to reset the scheduler error register in this function,
since we anyway clear the interrupt in fifo_error_isr()
Also, fix the typo
"reset initated" -> "reset initiated"
Bug 200089043
Change-Id: Ibfc2fd2133982a268699d08682bcd6f044a3196a
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/722711
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
Add platform specific API pointer (*get_iova_addr)()
which can be used to get iova/physical address from
given scatterlist and flags
Use this API with g->ops.mm.get_iova_addr() instead
of calling API gk20a_mm_iova_addr() which makes it
platform specific
Bug 1605653
Change-Id: I798763db1501bd0b16e84daab68f6093a83caac2
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/713089
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
There were still a couple of places using sg_phys directly. Use new
gk20a_mem_phys() to make the code shorter.
Change-Id: I6eb9b14e0c14a27ec39bacd06ab24e31e99769ca
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/717502
Introduce mem_desc, which holds all information needed for a buffer.
Implement helper functions for allocation and freeing that use this
data type.
Change-Id: I82c88595d058d4fb8c5c5fbf19d13269e48e422f
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/712699
Dump GR status registers in case of ctxsw timeout.
This is helpful in case where ctxsw timeout is encountered
during stress testing but we lose the bad state since we do
the recovery. So dump as much status as we can when timeouts
are seen
Bug 200062436
Change-Id: Ie7d320cefa7b272f2cc607cdb5c01ba1f43ba1f2
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/708465
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
Make recovery a more straightforward process. When we detect a fault,
trigger MMU fault, and wait for it to trigger, and complete recovery.
Also reset engines before aborting channel to ensure no stray sync
point increments can happen.
Change-Id: Iac685db6534cb64fe62d9fb452391f43100f2999
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/709060
(cherry picked from commit 95c62ffd9ac30a0d2eb88d033dcc6e6ff25efd6f)
Reviewed-on: http://git-master/r/707443
Retrieve number of physical page bits based on chip.
Bug 1567274
Change-Id: I5a0f6a66be37f2cf720d66b5bdb2b704cd992234
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/601700
Query interrupt number and reset id from HW. Use the number
from HW when enabling and detecting interrupts.
Bug 200036089
Bug 1567274
Change-Id: If9cb4db79a19dcb193ba7ad9db7081f4fe1ab433
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/600988
Do not call PMU ELPG calls if ELPG should be disabled. Also skips
initialization of PMU ucode if PMU is disabled.
Bug 1567274
Change-Id: Ia9cd3b553c358142ee05a1b0e0832f9412f7cf17
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/593335
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Fix below sparse warnings :
warning: Using plain integer as NULL pointer
warning: symbol <variable/funcion> was not declared. Should it be static?
warning: Initializer entry defined twice
Also, remove dead functions
Bug 1573254
Change-Id: I29d71ecc01c841233cf6b26c9088ca8874773469
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/593363
Reviewed-by: Amit Sharma (SW-TEGRA) <amisharma@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Sachin Nikam <snikam@nvidia.com>
With CG enabled sometimes fifo could not be idled
during firmware load.
Bug 200042729
Change-Id: I43d7551c0c7c19314c52ac5f678afed8c6df6415
Signed-off-by: Vijayakumar <vsubbu@nvidia.com>
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Signed-off-by: Vijayakumar <vsubbu@nvidia.com>
Reviewed-on: http://git-master/r/559077
Reviewed-by: Automatic_Commit_Validation_User
Convert GR and LTC HALs to use const structs, and initialize them
with macros.
Bug 1567274
Change-Id: Ia3f24a5eccb27578d9cba69755f636818d11275c
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/590371
When initializing fifo, we ignore several error conditions. Add
checks for them.
Change-Id: Id67f3ea51e3d4444b61a3be19553a5541b1d1e3a
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/553269
If engine status is in context switch in the fifo mmu fault handler,
dump falcon stats and gr stats for each engine.
Bug 1544766
Change-Id: Idfa9772b7e67072941144ac3bdd73e791fdc2b23
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/553205
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
To help remove the nvhost dependency from nvgpu, rename ioctl defines
and structures used by nvgpu such that nvhost is replaced by nvgpu.
Duplicate some structures as needed.
Update header guards and such accordingly.
Change-Id: Ifc3a867713072bae70256502735583ab38381877
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/542620
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
Retrieve channel count from gm20b specific header instead of the
gk20a header. This increases channel count from 128 to 512.
Change-Id: I96d4887432852795f7f526e33f0d3d2458f3af0e
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/500623
Invalid method needs to be cleared in gm20b to prevent getting same
interrupt again.
Change-Id: I4d83d1a27e5c711b5d82b95552be84d5f16a13e0
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/500286
Runlist event is not sent in gm20b for updated runlist. Polling is
the preferred way also for gk20a.
Bug 1555239
Change-Id: I60de084db69f848f63451f1f3078f183ca51ba50
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/500241
Use TSG specific API gk20a_fifo_recover_tsg() in following cases :
- IOCTL_CHANNEL_FORCE_RESET
to force reset a channel in TSG, reset all the channels
- handle pbdma intr
while resetting in case of pbdma intr, if channel is part of
TSG, recover entire TSG
- TSG preempt failure
when TSG preempt times out, use TSG recover API
Use preempt_tsg() API to preempt if channel is part of TSG
Add below two generic APIs which will take care of preempting/
recovering either of channel or TSG as required
gk20a_fifo_preempt()
gk20a_fifo_force_reset_ch()
Bug 1470692
Change-Id: I8d46e252af79136be85a9a2accf8b51bd924ca8c
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/497875
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
- add support to handle MMU faults on a channel in TSG
- first get the ID and type of channel that engine is running
- if TSG, abort each channel in it
- if regular channel, abort that channel
- also, add two versions of API set_ctx_mmu_error(), one for
regular channel and another for TSG
Bug 1470692
Change-Id: Ia7b01b81739598459702ed172180adb00e345eba
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/497874
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
- add and export API "gk20a_fifo_recover_tsg()" to
recover a TSG
- if TSG is running on any engine, then trigger MMU fault
on those engines
- otherwise, abort each channel in TSG
- modify channel specific API engines_on_ch() to generic
engines_on_id() which will take an ID and a flag to specify
whether ID is for channel or TSG and return engines running
on that ID
- modify channel specific API get_faulty_channel() to generic
get_faulty_id_type() which will take pointers to ID and type
of ID (either a regular channel or TSG)
- remove runlist update from recover_ch() since
no need to touch runlist during recovery
- set error notifier first and then only abort the channels
for TSG recovery path
- also, add necessary accessors to get engine
status type as TSG
Bug 1470692
Change-Id: I7137f611f80916b3d256d4b0dc6e5cf1e93eef6f
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/497873
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
If a channel is already scheduled as regular channel, we should not
allow it to be marked as TSG since it will fail book keeping of
number of active channels in a TSG
This way we can force to bind the channels first and then only
make them active
Also, remove duplicate function declaration added during branch merge
and one unnecessary comparison with zero
Bug 1470692
Change-Id: I88f9678919e4b76de472c6dda21e4537520241c4
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/497903
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Add API gk20a_fifo_preempt_tsg() which takes ID of tsg
and preempts it
Bug 1514064
Bug 1470692
Change-Id: I1d52c1dd7a9aecc1314b0f223fe4eedecc033629
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/495583
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Regenerate clock gating lists. Add new blocks, and takes them into
use. Also moves some clock gating settings to be applied at the
earliest possible moment right after reset.
Change-Id: I21888186c200f7a477c63bd3332e8ed578f63741
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/457698
The nvgpu driver now supports using the Tegra graphics virtualization
interfaces to support gk20a in a virtualized environment.
Bug 1509608
Change-Id: I6ede15ee7bf0b0ad8a13e8eb5f557c3516ead676
Signed-off-by: Aingara Paramakuru <aparamakuru@nvidia.com>
Reviewed-on: http://git-master/r/440122
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
