When kernel adds patches to a context, kernel needs to update
the patch count in order for FECS to pick up the new patches.
Previously patching was done only at the context creation
time. Now patching is used also when changing preemption mode,
but the patches did not take effect due to not updating count.
Update patch count every time we end patching of a context.
Bug 1852094
Change-Id: Ic2150741609d1d1956769e439ce1c5f2edcacb84
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1280424
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Reorganize the HW headers of gk20a. The headers are moved to a
new directory:
include/nvgpu/hw/gk20a
And from the code are included like so:
#include <nvgpu/hw/gk20a/hw_pwr_gk20a.h>
This is the first step in reorganizing all of the HW headers for
gm20b, gm206, etc. This is part of a larger effort to re-structure
and make the driver more readable and scalable.
Bug 1799159
Change-Id: Ic151155cbc2e6f75009f2d9d597b364a1bed2c4c
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1244790
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
If enabled, track actions (gets and puts) on channel reference counters.
Dump the most recent actions to syslog when
gk20a_wait_until_counter_is_N gets stuck when closing a channel.
GK20A_CHANNEL_REFCOUNT_TRACKING specifies the size of the action
history. Default is to disable completely, as this has some runtime
overhead.
Bug 1826754
Change-Id: I880b0efe8881044d02ae224c243a51cb6c2db8c1
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1262424
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
- Added struct pmu_pg_stats_data to extract
data from multiple version of pmu pg statistics
- Added pmu_pg_stats_v2 interface to fetch
PG statistics data from PMU
- Added MSCG debugfs node to read mscg
statistics from PMU.
- Added pmu_elpg_statistics HAL support for
gp106 PG statistics read.
- Made changes to gp104/gp106
pmu_elpg_statistics HAL to support
for struct pmu_pg_stats_data
JIRA DNVGPU-165
Change-Id: I2b9e89c0fae90deb45006c4478170b9a97b56603
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: http://git-master/r/1252798
(cherry picked from commit 3c073b15fd991db8d65b3171b02c161294be40cd)
Reviewed-on: http://git-master/r/1271615
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Kickoff latencies when GPFIFO is in vidmem grow significantly as
function of number of GPFIFO entries. Move GPFIFO to sysmem to
improve kickoff latency.
Bug 1848369
Change-Id: Ie95d10df26b4f1370f7250a8fbf0f7ef0211df32
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
(cherry-picked from commit 897cb579a759bbe8455ce368413979e91eb0d475)
Reviewed-on: http://git-master/r/1281564
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Move the GPU allocators to common/mm/ since the allocators are common
code across all GPUs. Also rename the allocator code to move away from
gk20a_ prefixed structs and functions.
This caused one issue with the nvgpu_alloc() and nvgpu_free() functions.
There was a function for allocating either with kmalloc() or vmalloc()
depending on the size of the allocation. Those have now been renamed to
nvgpu_kalloc() and nvgpu_kfree().
Bug 1799159
Change-Id: Iddda92c013612bcb209847084ec85b8953002fa5
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1274400
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
In gk20a_busy() we check if dev is NULL and return -ENODEV if so. But
before that we've already dereferenced dev by passing it to
get_gk20a(). Defer call to get_gk20a() until after the NULL check.
Bug 200192125
Change-Id: I943a9e96d13ff8cb4333fe20a941c8e95d159a66
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1280349
GVS: Gerrit_Virtual_Submit
Add a platform flag and logic to disable ASPM on boot if a
platform does not want ASPM enabled.
Bug 200256272
Change-Id: I378997290377ef8ffa21bf8e7f3f59fa134b3d4d
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1263271
(cherry picked from commit 01e4700b74f4b4c6f0b9ffb40747653f2b63ea3c)
Reviewed-on: http://git-master/r/1274477
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Handle possible asynchronous GPU driver shutdown more gracefully.
This occurs when the GPU disappears from the PCI bus, for example,
if it overheats or detects an over current event.
Also add a preprocessor check to make sure that the
gk20a_channel_cancel_pending_sema_waits()
is always defined. In some builds CONFIG_SYNC is disabled but the
gk20a_remove_support() code does not check for this.
Bug 1816516
Bug 1807277
Change-Id: I932e312291c5c6a6ac5e13525ce8ca56a1be3652
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1250028
(cherry picked from commit 337810f8c478238a38d8553c1492622d5fa9aafa)
Reviewed-on: http://git-master/r/1274476
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Move the code that waits for deferred interrupts to nvgpu_common.c and
make it global. Also rename that function to use nvgpu_ as the function
prefix.
Bug 1816516
Bug 1807277
Change-Id: I42c4982ea853af5489051534219bfe8b253c2784
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1250027
(cherry picked from commit cb6fb03e20b08e5c3606ae8a5a9c237bfdf9e7da)
Reviewed-on: http://git-master/r/1274475
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Fix miscellaneous issues seen during driver shutdown.
o Make sure pointers are valid before accessing them.
o Busy the GPU during channel timeout.
o Cancel delayed work on channels.
o Avoid access to channels that may have been freed.
Bug 1816516
Bug 1807277
Change-Id: I62df40373fdfb1c4a011364e8c435176a08a7a96
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1250026
(cherry picked from commit 64a95fc96c8ef7c5af9c53c4bb3402626e0d2f60)
Reviewed-on: http://git-master/r/1274474
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
In many helper functions like gk20a_readl() the code assumed that
the GPU is present and registers and available. However, during
GPU shutdown this may not be the case.
In theory the driver should not be accessing GPU registers during
GPU shutdown (since shutdown is triggered by GPU registers being
unavailable) but these changes handle any missed cases where this
may happen.
This goes for GPU device access as well. Many parts of the code
assume that if the struct gk20a is valid, the the GPU dev must be
there are well. This isn't always the case, it seems.
Bug 1816516
Bug 1807277
Change-Id: Icaf6fd56ab7860724e77bda0f5e8d48f0da15642
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1250024
(cherry picked from commit e8c9997b2d7cd424d798ecfce1307e6193c0cf32)
Reviewed-on: http://git-master/r/1274473
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Enable FECS ctxsw tracing for gp106. Ensure that FECS records
have been written to memory before accessing the ring. Update
read index only once all records have been processed.
Jira EVLR-424
Change-Id: I1a21f841fcce1588397408906d77e2c3bf4a8c01
Signed-off-by: Thomas Fleury <tfleury@nvidia.com>
Reviewed-on: http://git-master/r/1258243
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
We use GPU reference clock as a divider. Check before division that
reference clock is not zero.
Change-Id: Ie453a78b422b2e740daeb7c12ce5b06faa52ba76
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1275743
GVS: Gerrit_Virtual_Submit
Reviewed-by: Thomas Fleury <tfleury@nvidia.com>
In gk20a_submit_prepare_syncs(), after we have allocated wait_cmd
we check the results. If we failed to allocate wait_cmd, we still
jump to the error label that tries to free wait_cmd.
Create an own error labal for allocation before wait_cmd, and use
that if we fail to allocate wait_cmd.
Similarly create an error label for incr_cmd, and use that only once
incr_cmd has actually been allocated.
Change-Id: I1f8bc1d947c524038f5f237358a5e6b0dc2e6ac3
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1275742
GVS: Gerrit_Virtual_Submit
When allocating job channel_gk20a_job structure we assign the result
of allocation to *job_out, but we check the result of allocation
in job_out.
Change the check to check for result in *job_out.
Change-Id: Ia170cfa2dd5730665434b4c223c5a2f9502c744d
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1275741
GVS: Gerrit_Virtual_Submit
Reviewed-by: Thomas Fleury <tfleury@nvidia.com>
We use &ch->error_notifier_mutex to protect
writes and free of error notifier
But we currently do not protect reading of
notifier in gk20a_fifo_set_ctx_mmu_error()
and vgpu_fifo_set_ctx_mmu_error()
Add new API gk20a_set_error_notifier_locked()
which is same as gk20a_set_error_notifier()
but without the locks.
In *_fifo_set_ctx_mmu_error() APIs, acquire
the mutex explicitly, and then use this new
API
gk20a_set_error_notifier() will now just call
gk20a_set_error_notifier_locked() within
a mutex
Bug 1824788
Bug 1844312
Change-Id: I1f3831dc63fe1daa761b2e17e4de3c155f505d6f
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1273471
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Richard Zhao <rizhao@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Process long regops lists in 4-kB fragments, overcoming the overly
low limit of 128 reg ops per IOCTL call. Bump the list limit to 1024
and report the limit in GPU characteristics.
Bug 200248726
Change-Id: I3ad49139409f32aea8b1226d6562e88edccc8053
Signed-off-by: Sami Kiminki <skiminki@nvidia.com>
Reviewed-on: http://git-master/r/1253716
(cherry picked from commit 22314619b28f52610cb8769cd4c3f9eb01904eab)
Reviewed-on: http://git-master/r/1266652
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
- PG statistics read support for multiple engines
- updated stat_dmem_offset member to array to hold
dmem offset of PG engines
- PMU allocates memory in DMEM for each PG engine requested,
updated gk20a_pmu_get_elpg_residency_gating() to get
engine statistics for requested PG engine
JIRA DNVGPU-71
Change-Id: I2ddade37f85716f757bf33034dbff816184577eb
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: http://git-master/r/1250506
(cherry picked from commit 68ba7a97d6662b87d0e489365d8afb8e2d237a03)
Reviewed-on: http://git-master/r/1270972
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
- Added enable_mscg, mscg_enabled & mscg_stat flags,
mscg_enabled flag can be used to controll
mscg enable/disable at runtime along with mscg_stat flag.
- Added defines & interface to support ms/mclk-change/post-init-param
- Added defines for lpwr tables read from vbios.
- HAL to support post init param which is require
to setup clockgating interface in PMU & interfaces used during
mscg state machine.
- gk20a_pmu_pg_global_enable() can be called when pg support
required to enable/disable, this also checks & wait
if pstate switch is in progress till it complets
- pg_mutex to protect PG-RPPG/MSCG enable/disable
JIRA DNVGPU-71
Change-Id: If312cefc888a4de0a5c96898baeaac1a76e53e46
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: http://git-master/r/1247554
(cherry picked from commit e6c94948b8058ba642ea56677ad798fc56b8a28a)
Reviewed-on: http://git-master/r/1270971
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
graphics and compute preemption modes are currently
defined as int
But it is more logical to have them as unsigned int
Also, we treat preemption modes as unsigned almost
everywhere in the code
Fix prints in gk20a_fifo_sched_debugfs_seq_show() to
print U32_MAX with %d which is same as printing -1
Bug 200263471
Change-Id: Iabd0ee3923b76d81620898e90a9b1fc5dd75b530
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1272514
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Fix use-after-free overity defect in page allocator. The alloc struct
was getting used after a call for __gk20a_free_pages() which frees
the alloc struct passed in.
Coverity ID: 468942
Bug 200192125
Change-Id: I4f5d32f245efae967050f93c7806290b4bf3591c
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1272730
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Move the sw initialization of the gpcclk from gpu rail ungate path
to the nvgpu probe path. This allows gpcclk to register itself
successfully with CCF and makes it discoverable for other clients
early on during boot.
Bug 200233943
Bug 200259437
Change-Id: I88d94542092f92e68dc63c40444a70991d1f6129
Signed-off-by: Srikar Srimath Tirumala <srikars@nvidia.com>
Reviewed-on: http://git-master/r/1265549
Reviewed-by: Tejal Kudav <tkudav@nvidia.com>
- pmu_init_powergating loops & init multiple
PG engines based on PG engines supported
- generalize pg init param HAL to support
multiple PG-engine init based on PG engine
parameter
- HAL's to return supported PG engines on chip &
its sub features of engine.
- Send Allow/Disallow for PG engines which are
enabled & supported.
- Added defines for pg engines
JIRA DNVGPU-71
Change-Id: I236601e092e519a269fcb17c7d1c523a4b51405f
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: http://git-master/r/1247409
(cherry-picked from commit 1c138cc475bac7d3c3fbbd5fb18cfcb2e7fdf67a)
Reviewed-on: http://git-master/r/1269319
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Change "check" to "expired" in nvgpu_timeout_check* and append _expired
to nvgpu_timeout_peek to clarify what the boolean-like return value
means and thus avoid bugs.
Bug 200260715
Change-Id: I47e097ee922e856005a79fa9e27eddb1c8d77f8b
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1269366
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
When the user-supplied ioctl argument size is too large, just return
-EINVAL from the ioctl instead of crashing on a BUG_ON (for as, ctrl,
ctxsw, dbg and tsg nodes - channel and sched nodes are already okay).
Bug 1849661
Change-Id: I5b0d1d0c4ee47ce0136c424dda5975353f110c7e
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1266606
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Store pending sema waits so that they can be explicitly handled when
the driver dies. If the sema_wait is freed before the pending wait is
either handled or canceled problems occur.
Internally the sync_fence_wait_async() function uses the kernel timers.
That uses a linked list of possible events. That means every so often
the kernel iterates through this list. If the list node that is in the
sync_fence_waiter struct is freed before it can be removed from the
pending timers list then the kernel timers list can be corrupted. When
the kernel then iterates through this list crashes and other related
problems can happen.
Bug 1816516
Bug 1807277
Change-Id: Iddc4be64583c19bfdd2d88b9098aafc6ae5c6475
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1250025
(cherry picked from commit 01889e21bd31dbd7ee85313e98079138ed1d63be)
Reviewed-on: http://git-master/r/1261920
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Ensure that any open channel is definitely closed before freeing it.
Bug 1816516
Bug 1807277
Change-Id: I7f100db5ab6834176ec97d22374646d3336f2856
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1250023
(cherry picked from commit 56f1b4b4312c5900f1c27eba55ad970c4b264f24)
Reviewed-on: http://git-master/r/1261919
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Allow forced channel freeing. This is useful when the driver is
being cleaned up and the gk20a_wait_until_counter_is_N() could
potentially hang.
Bug 1816516
Bug 1807277
Change-Id: I711f5f3f6413d0bb30b4857e785ca3b504b494ee
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1250022
(cherry picked from commit e132d0e5ae77d758680ac708622a4883bbd69ba3)
Reviewed-on: http://git-master/r/1261918
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Check if the GPU is present after each register read. If the a register
read returns 0xffffffff then it's possible the GPU has fallen off the
bus for some reason or another. However, to confirm that a register read
is due to a dead GPU vs just a 0xffffffff being returned by happenstance
the chip ID register is read which should never return 0xffffffff. If
that read returns 0xffffffff as well then certainly the GPU is dead.
Bug 1805082
Bug 1816516
Bug 1807277
Change-Id: I4de61b56289217d9c0d8167e84615a67c8bde8a9
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1239518
(cherry picked from commit bd50828de20aba9b2887ee99c2269602c21a793f)
Reviewed-on: http://git-master/r/1261916
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Allow SW to force a semaphore release. Typically SW waits for a
semaphore to reach the value right before the semaphore release
value before doing a release. For example, assuming a semaphore
is to be released by SW by writing a 10 into the semaphore, the
code waits for the semaphore to get to 9 before writing 10.
The problem with this happens when trying to shutdown the GPU
unexpectedly. When aborting a channel after the GPU has terminated
the GPU is potantially no longer processing anything. If a SW
semaphore release is waiting on the semaphore to reach N-1 before
writing N to the semaphore N-1 may never get written by the GPU.
This obviously causes a hang in the SW shutdown. The solution is
to let SW force a semaphore release in the channel_abort case.
Bug 1816516
Bug 1807277
Change-Id: Ib8b4afd86102eacf372362b1748fb6ca04e6fa66
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1250021
(cherry picked from commit 2e9fa40902d2c4d5a1febe0bf2db420ce14bc633)
Reviewed-on: http://git-master/r/1261915
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Add a private tsg struct for the TSG file descriptors. This allows
the TSG files to retain access to the gk20a struct without the TSG
data structure still being available.
Bug 1816516
Bug 1807277
Change-Id: If0e41d912bebb7906d72e7eee641f47604a16494
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1250020
(cherry picked from commit 5d8dbbb939e10fecde341891807a0201a63b2b23)
Reviewed-on: http://git-master/r/1261914
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
The private_data field in the file pointer passed to release() for
channels originally pointed directly to the referenced channel. The
problem with this is that when the driver is killed and the channel
mmeory is freed that pointer becomes invalid.
The necessity of that channel is to get access to the gk20a struct that
owns the channel. This can instead be accomplished by making a new
private data struct that has a pointer to the gk20a struct directly
instead of requiring the channel to be valid. This lets the release()
function work even if the channels are gone (though in such cases the
release function doesn't do very much).
Change-Id: I5e50bb5b6dd08d38974f8e7b46ba125e9a3f1922
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1246586
(cherry picked from commit 14b7c380c74d2caeb04c47ad3e33332a423a84bb)
Reviewed-on: http://git-master/r/1261913
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
