Wait for pbdma and engine to go idle so that the tasks get completed before
suspending.
Updated the logic in gk20a_wait_engine_idle to consider the ctxsw status.
And updated PBDMA idle logic to check the pbdma status and the pb/gp
get/put pointers.
Bug 3789519
Bug 3832838
Change-Id: Ifd105bbb305eaf358423281b192f67d782d773a4
Signed-off-by: Sagar Kamble <skamble@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2870162
Reviewed-by: Martin Radev <mradev@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
GVS: Gerrit_Virtual_Submit <buildbot_gerritrpt@nvidia.com>
To support concurrent UMD kickoff and railgate (due to VPR resize), It
is necessary that nvgpu immediately prevents more work submission when
it's known gpu needs to be idled. So nvgpu needs to unmap the usermode
region earliest during suspend sequence.
Otherwise, engines will not go idle before poweroff. Hence move the
call to nvgpu_hide_usermode_poweroff to the beginning of
gk20a_pm_prepare_poweorff.
Also during suspend we ensure that the channels are preempted cleanly.
IRQs should be kept enabled until after channels are suspended as the
stalling IRQ can block the preemption. Hence moved the IRQ disable
post channel_suspend call.
gk20a_prepare_poweroff unconditionally sets power_on to false. Hence
there is no need to reenable IRQs, resume scale in the failure path
of gk20a_pm_prepare_poweroff as those will be done during call to
gk20a_pm_finalize_poweron.
Bug 3789519
Change-Id: I03064e7e636252a8f3d8fe9c8c05629ce2ba5fba
Signed-off-by: Sagar Kamble <skamble@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2853584
Reviewed-by: Martin Radev <mradev@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
GVS: Gerrit_Virtual_Submit <buildbot_gerritrpt@nvidia.com>
UMD is unable to restrict railgate for deterministic channels
when NVGPU_CAN_RAILGATE is set to false. This causes issues
with VPR resize as there is no means of preventing an active
VPR resize in progress.
Add a fault handler for usermode region. The fault handler's purpose is
to intercept UMD accesses into the doorbell region when a GPU reset
is in progress. GPU reset could be triggered by VPR resize. During a
reset, the corresponding PTEs for the usermode region are zapped. The
fault handler tries to have a read access to g->deterministic_busy
and blocks till the reset is finished. A VPR resize is guaranteed
to be mutually exclusive due to use of the g->deterministic_busy
RW semaphore.
Bug 3789519
Change-Id: Ie046ee9be8d9b5d4019359c60a4578097b8d55a3
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
Signed-off-by: Sagar Kamble <skamble@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2802185
Reviewed-by: Martin Radev <mradev@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
GVS: Gerrit_Virtual_Submit <buildbot_gerritrpt@nvidia.com>
%p prints (ptrval) instead of hexadecimal value until it gathers
enough entropy. During early boot stage, it can make invalid
cache name like "nvgpu-cache-0x (ptrval)-128-1" and that
kind of cache name can make failure of kmem_cache_create().
To avoid invalid cache name, replace %p with %px for cache name.
Bug 4100509
Change-Id: Iae0ae9cf1a30ec91aeddddaafda9e7376fc80796
Signed-off-by: Jake Park <jakep@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2929270
Reviewed-by: Dmitry Pervushin <dpervushin@nvidia.com>
Reviewed-by: Kwangwoo Lee <kwangwool@nvidia.com>
GVS: Gerrit_Virtual_Submit <buildbot_gerritrpt@nvidia.com>
gk20a_scale_target is called through the target member of a
devfreq_profile. It is only called from devfreqs update_devfreq
function or through governor_passive. governor_passive is not used for
nvgpu.
Since update_devfreq already enforce the devfreq limits,
gka20_scale_target can be simplified by only checking pm_qos
limits and also only if GK20A_PM_QOS is enabled.
This also resolves a race between creating devfreq sysfs
files and setting 'l->devfreq' in gk20a_scale_init that can
lead to accessing a NULL pointer by writing to the sysfs files.
Example:
Unable to handle kernel NULL pointer dereference at virtual address 00000430
<snip>
Call trace:
[<000000006aa50d89>] gk20a_scale_target+0x5c/0x120 [nvgpu]
[<00000000e5a63f7c>] update_devfreq+0xec/0x22c
[<0000000014a13c8a>] max_freq_store+0xa8/0xfc
[<0000000072139393>] dev_attr_store+0x48/0x60
[<000000008ec280df>] sysfs_kf_write+0x60/0x70
[<0000000038427ed5>] kernfs_fop_write+0xc4/0x1e0
[<00000000c0b74aa9>] __vfs_write+0x60/0x14c
[<0000000078fcebb4>] vfs_write+0xb0/0x1b4
[<000000007720da30>] SyS_write+0x74/0xf0
[<0000000067443e2c>] __sys_trace_return+0x0/0x4
Bug 3910155
Change-Id: I7193cc5ea85454acf0890b3ca8d1c3526ca8517e
Signed-off-by: Ken Chang <kenc@nvidia.com>
Signed-off-by: Sagar Kamble <skamble@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2828219
Reviewed-by: Rajesh Devaraj <rdevaraj@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
On volta the GPU determines whether to do L3 allocation for a mapping by
checking bit 36 of the physical address. So if a mapping should allocate lines
in the L3 this bit must be set.
However, when the physical addresses for 64GB of RAM uses the 36th bit
resulting in a conflict. Thus, add support for disabling l3 support
for SKUs having 64GB of physical memory.
Bug 3486025
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
Change-Id: Ic540e754274cf1d9e6625493962699d21509e540
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2661548
Reviewed-by: Brad Griffis <bgriffis@nvidia.com>
Reviewed-by: Bibek Basu <bbasu@nvidia.com>
Tested-by: Brad Griffis <bgriffis@nvidia.com>
GVS: Gerrit_Virtual_Submit
clk_arb completion file descriptor can get closed immediately after
poll finishes in the work item gp10b_clk_arb_run_arbiter_cb. In
that case, the refcount for nvgpu_clk_dev can become zero in
the work item and can lead to invalid access while removing
nvgpu_clk_dev from the lists.
Remove nvgpu_clk_dev from the list before dropping the reference to
it.
Also, delete the nvgpu_clk_dev in completion file release handler
within the session and requests spinlocks to avoid race with
gp10b_clk_arb_run_arbiter_cb using it.
bug 200757277
Change-Id: I054eee547f2a6fa633d7ef55df216ec36647a826
Signed-off-by: Sagar Kamble <skamble@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2569522
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Reviewed-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-by: Bibek Basu <bbasu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Implement nvgpu plumbing to allow reporting ECC errors(corrected
and uncorrected) to a L1SS service(if one exists).
This patch includes the following
1) Added code that submits ECC error reports via the Interrupt context
directly to a L1SS service in linux OS.
2) Added support for enabling/disabling the error reports via L1SS's
registration/deregistration API. Nvgpu simply invokes an empty function
until the registration is successful.
3) Added Spinlock to correctly handle concurrency for accessing the
correct Ops for submitting requests.
4) Adds error reporting for a subset of interrupts that can be verified
via external ECC injection logic. A subsequent patch will add the
API for rest of the interrupts.
5) In case of critical(uncorrected errors), change nvgpu's state to
quiesce state.
Jira L4T-1187
Bug 200700400
Change-Id: Id31f70531fba355e94e72c4f9762593e7667a11c
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2530411
Tested-by: Bibek Basu <bbasu@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Reviewed-by: Bibek Basu <bbasu@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
GVS: Gerrit_Virtual_Submit
Below change added capability check in the ioctl. nvgpu is advertising
the support for RESCHEDULE_RUNLIST for all processes even though it
fails the ioctl for non-realtime processes.
Clear the ioctl flag for RESCHEDULE_RUNLIST for non-realtime processes.
commit 838ba0a14d ("gpu: nvgpu: check capability for reschedule runlist submit flag")
Author: David Li <davli@nvidia.com>
Date: Tue Sep 12 18:37:00 2017 -0700
NVGPU_SUBMIT_GPFIFO_FLAGS_RESCHEDULE_RUNLIST is only used by realtime
priority EGL context, which checks for CAP_SYS_NICE during context
creation in userspace, so it wasn't secure against unprivileged program
spoofing submit ioctl with this flag to stall GPU progress of others.
This flag does increase duration of submit by approx 16us,
mostly due to register accesses and PMU FIFO mutex.
Bug 2823941
Change-Id: Iecee3989e5af035264b1ed5c1aa9a8576dd90883
Signed-off-by: Sagar Kamble <skamble@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2372957
(cherry picked from commit 864213ae55b009b0a026ac380b26276332f79177)
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2392714
Reviewed-by: automaticguardword <automaticguardword@nvidia.com>
Reviewed-by: Bibek Basu <bbasu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
GVS: Gerrit_Virtual_Submit
Currently, for platforms with canRailgate device characteristics disabled,
suspend can block as deterministic channels hold busy references. This
patch makes the change to first hold off any new jobs for deterministic
channels and then reverts back the busy references taken by those
channels. Following this, suspend also waits for the device to get idle
by waiting (with timeout) for the nvgpu's internal usage counter to be
come zero. This ensures there are no further jobs in progress and
allows the system to go into a suspend state.
Bug 200598228
Bug 2930266
Change-Id: Id02b4d41a9c2dd64303b2e2449dbed48c12aea4c
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2328489
Reviewed-by: automaticguardword <automaticguardword@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-by: Shashank Singh <shashsingh@nvidia.com>
Reviewed-by: Sami Kiminki <skiminki@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Bibek Basu <bbasu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
GVS: Gerrit_Virtual_Submit
Background: There is a race that occurs when l2_fb_ops ioctl is
invoked. The race occurs as part of the flush() call while a
gk20_idle() is in progress.
This patch handles the race by making changes in the l2_fb_ops
ioctl itself. For cases where pm_runtime is disabled or railgate is
disabled, we allow this ioctl call to always go ahead as power is
assumed to be always on.
For the other case, we first check the status of g->power_on. In the
driver, g->power_on is set to true, once unrailgate is completed and is
set to false just before calling railgate.
For linux, the driver invokes gk20a_idle() but there is a delay after
which the call to the rpm_suspend()'s callback gets triggered. This
leads to a scenario where we cannot efficiently rely on the
runtime_pm's APIs to allow us to block an imminent suspend or exit if
the suspend is currently in progress. Previous attempts at solving this
has lead to ineffective solutions and make it much complicated to
maintain the code.
With regards to the above, this patch attempts to simplify the way this
can be solved. The patch calls gk20a_busy() when g->power_on = true.
This prevents the race with gk20a_idle(). Based on the rpm_resume and
rpm_suspend's upstream code, resume is prioritized over a suspend
unless a suspend is already in progress i.e. the delay period has been
served and the suspend invokes the callback. There is a very small
window for this to happen and the ioctl can then power_up the device as
evident from the gk20a_busy's calls.
A new function gk20a_check_poweron() is added. This function protects
the access to g->power_on via a mutex. By preventing a read from
happening simulatenously as a write on g->power_on, the likelihood of
an runtime_suspend triggering before a runtime_resume is further
reduced.
Bug 200507468
Change-Id: I5c02dfa8ea855732e59b759d167152cf45a1131f
Signed-off-by:Debarshi Dutta <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2299545
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Replaced ch->mmu_debug_mode_enabled with ch->mmu_debug_mode_refcnt.
If channel is enabled multiple times by userspace, then ref count is
updated accordingly. There is an expectation that enable/disable
calls are balanced for setting channel's mmu debug mode.
When unbinding the channel, decrease refcnt for the channel until it
reaches 0.
Also, removed tsg parameter from nvgpu_tsg_set_mmu_debug_mode as it
can be retrieved from ch.
Bug 2515097
Bug 2713590
Change-Id: If334e374a55bd14ae219edbfd3b1fce5ff25c226
Signed-off-by: Thomas Fleury <tfleury@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/2184702
(cherry picked from commit f422aee393)
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2208772
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Kajetan Dutka <kdutka@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Winnie Hsu <whsu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: Kajetan Dutka <kdutka@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
GVS: Gerrit_Virtual_Submit
Added NVGPU_DBG_GPU_IOCTL_SET_CTX_MMU_DEBUG_MODE ioctl to set MMU
debug mode for a given context.
Added gr.set_mmu_debug_mode HAL to change NV_PGPC_PRI_MMU_DEBUG_CTRL
for a given channel. HAL implementation for native case is
gm20b_gr_set_mmu_debug_mode. It internally uses regops, which directly
writes to the register if the context is resident, or writes to
gr context otherwise.
Added NVGPU_SUPPORT_SET_CTX_MMU_DEBUG_MODE to enable the feature.
NV_PGPC_PRI_MMU_DEBUG_CTRL has to be context switched in FECS ucode,
so the feature is only enabled on TU104 for now.
Bug 2515097
But 2713590
Change-Id: Ib4efaf06fc47a8539b4474f94c68c20ce225263f
Signed-off-by: Thomas Fleury <tfleury@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/2110720
(cherry-picked from commit af2ccb811d)
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2208767
Reviewed-by: Kajetan Dutka <kdutka@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Winnie Hsu <whsu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
GVS: Gerrit_Virtual_Submit
Tested-by: Kajetan Dutka <kdutka@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
When the TSG is initialized, the timeslice is initially set to 0,
which is a flag for using the default timeslice.
The sched get_params ioctl was returning 0 in this case; instead,
call gk20a_tsg_get_timeslice() to get the correct value.
Bug 2818547
Change-Id: I29fde48240098074de12e9a9a3e347e4042a64e8
Signed-off-by: Peter Daifuku <pdaifuku@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2283962
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Thomas Fleury <tfleury@nvidia.com>
Reviewed-by: Satish Arora <satisha@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
GVS: Gerrit_Virtual_Submit
gk20a_idle can still race with gk20a_busy_noresume as the following
pm_runtime's functions can run concurrently. i.e. pm_runtime_suspend and
pm_runtime_get_noresume. pm_runtime_get_noresume simply increments the
refcount without first acquiring the pm_runtime's power_lock. This can
be resolved by the use of pm_runtime_get_if_in_use which acquires a
the power lock of PM runtime. This prevents a potential use of register
after power_off in the l2_ops ioctl path.
We still call pm_runtime_get_noresume even if pm_runtime_get_if_in_use
returns <= 0. This helps in the following ways.
1) The signature of the function gk20a_busy_noresume remains same w.r.t
QNX code
2) Any calls to gk20a_busy_noresume() in the driver is followed by a
check to g->power_on. When pm_runtime_get_if_in_use() returns <=0,
g->power_on is false and would immediately call gk20a_suspend_noidle()
thus keeping the original 'intent' of the gk20a_busy_noresume intact.
Bug 200507468
Change-Id: Id46bf2124047bb0e9a299fda57441e425195468f
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/2180787
Reviewed-by: Bibek Basu <bbasu@nvidia.com>
Reviewed-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Writing same value to railgate_enable_store should be treated as nop
and made successfully. Doing so is not only an optimization for the
operation but also convention that users expect for "settings". This
change is primary for fixing a peculiar situation in the driver:
root@localhost:/sys/devices/17000000.gp10b# cat railgate_enable
0
root@localhost:/sys/devices/17000000.gp10b# echo 0 > railgate_enable
bash: echo: write error: Invalid argument
Attempt to disable railgating on a platform where railgating isn't
supported shouldn't be treated as 'invalid'. It's disabled after all.
Bug 200562094
Change-Id: I3c04934bdbaf337c33d7de9cac6d53c96b4dacae
Signed-off-by: Leon Yu <leoyu@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/2225476
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-by: Bibek Basu <bbasu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
This patch adds nvgpu API in linux and qnx to query vpr resize.
The new API nvgpu_is_vpr_resize_enabled() is used in
nvgpu_submit_channel_gpfifo().
Previously, if non-deterministic channel has timeout disabled and
GPU cannot railgate on some platform, then channel doesn't power ref
count and results in video freeze. This requires non-determinstic
channel job tracking to be enabled if vpr resize is supported or if GPU
can railgate.
Bug 200532122
Change-Id: Icfbff6253762b195b2f5955749343974b1a7a269
Signed-off-by: Vedashree Vidwans <vvidwans@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/2167082
Reviewed-on: https://git-master.nvidia.com/r/2180581
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Import userd and gpfifo buffers from userspace if provided via
NVGPU_IOCTL_CHANNEL_ALLOC_GPFIFO_EX. Also supply the work submit token
(i.e., the hw channel id) to userspace.
To keep the buffers alive, store their dmabuf and attachment/sgt handles
in nvgpu_channel_linux. Our nvgpu_mem doesn't provide such data for
buffers that are mainly in kernel use. The buffers are freed via a new
API in the os_channel interface.
Fix a bug in gk20a_channel_free_usermode_buffers: also unmap the
usermode gpfifo buffer.
Bug 200145225
Bug 200541476
Change-Id: I8416af7085c91b044ac8ccd9faa38e2a6d0c3946
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1795821
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
(cherry picked from commit 99b1c6dcdf
in dev-main)
Reviewed-on: https://git-master.nvidia.com/r/2170603
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bibek Basu <bbasu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
