Use struct nvgpu_mem for DMA allocations (and the corresponding
nvgpu_dma_alloc_sys()) instead of custom rolled code. This migrates
away from using linux scatter gather tables directly. Instead this
is hidden in the nvgpu_mem struct. With this change the semaphore.c
code no longer has any direct Linux dependencies.
JIRA NVGPU-12
JIRA NVGPU-30
Change-Id: I92167c98aac9b413ae87496744dcee051cd60207
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1464081
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Hide the Linux specific nvgpu_mem fields so that in subsequent patches
core code can instead of using struct sg_table it can use mem_desc.
Routines for accessing system specific fields will be added as needed.
This is the first step in a fairly major overhaul of the GMMU mapping
routines. There are numerous issues with the current design (or lack
there of): massively coupled code, system dependencies, disorganization,
etc.
JIRA NVGPU-12
JIRA NVGPU-30
Change-Id: I2e7d3ae3a07468cfc17c1c642d28ed1b0952474d
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1464076
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
The debugfs submit profiling stats report the distribution of submit
times at different percentiles. The original code reports the lowest
entry in every percentile-bin as being representative for that
bin. This means that where debugfs reports the 50th percentile, it
actually reports the ~45.001th percentile. This is especially
inconveniant when looking at the 100th percentile, which does not in
fact reflect the maximum.
This patch changes the code to instead report the highest entry in
every percentile-bin.
Change-Id: Ibde3aaf6445e08af09a4a96156f8421cbeb8bae5
Signed-off-by: Peter Boonstoppel <pboonstoppel@nvidia.com>
Reviewed-on: http://git-master/r/1461658
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Use platform agnostic delay functions instead of Linux kernel APIs.
This allows removing dependency to Linux header linux/delay.h. At the
same time remove #include lines for other unused Linux headers.
JIRA NVGPU-16
Change-Id: I46b9ccb80e0b67efb86ec85676e5a55ff835c0ec
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1460113
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Add nvgpu_* wrappers for determining if we're running in simulation
or silicon, and if we're running in hypervisor.
The new wrappers require struct gk20a pointer, and gk20a_fence_wait()
did not have access to one. Add struct gk20a pointer as the first
parameter.
JIRA NVGPU-16
Change-Id: I73b2b8f091ca29fb1827054abd2adaf583710331
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1331565
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Patch e1e059454a
removed the API gk20a_fifo_disable_all_engine_activity()
which disables activity on all engines
Re-add it since it is now needed in the shutdown path
to ensure engines are idle
Bug 200281010
Change-Id: Idf5860f7f5009f3fb875ce94b7d66007c866ac49
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1457755
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
gk20a_err() and gk20a_warn() require a struct device pointer,
which is not portable across operating systems. The new nvgpu_err()
and nvgpu_warn() macros take struct gk20a pointer. Convert code
to use the more portable macros.
JIRA NVGPU-16
Change-Id: Ia51f36d94c5ce57a5a0ab83b3c83a6bce09e2d5c
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1331694
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
GVS: Gerrit_Virtual_Submit
Rename the nvgpu DMA APIs from gk20a_gmmu_alloc* to nvgpu_dma_alloc*.
This better reflects the purpose of the APIs (to allocate DMA suitable
memory) and avoids confusion with GMMU related code.
JIRA NVGPU-12
Change-Id: I673d607db56dd6e44f02008dc7b5293209ef67bf
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1325548
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Make an nvgpu DMA API include file so that the intricacies of the
Linux DMA API can be hidden from the calling code.
Also document the nvgpu DMA API.
JIRA NVGPU-12
Change-Id: I7578e4c726ad46344b7921179d95861858e9a27e
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1323326
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Renaming was done with the following command:
$ find -type f | \
xargs sed -i 's/struct mem_desc/struct nvgpu_mem/g'
Also rename mem_desc.[ch] to nvgpu_mem.[ch].
JIRA NVGPU-12
Change-Id: I69395758c22a56aa01e3dffbcded70a729bf559a
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1325547
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Replace the last of the Linux kmem API usage with nvgpu kmem
calls instead. Several places are left alone - allocating the
struct gk20a in particular.
Also one function was updated in the clk code to take a struct
gk20a as an argument so that it could use nvgpu_kmalloc().
Bug 1799159
Bug 1823380
Change-Id: I84fc3f8e19c63d6265bac6098dc727d93e3ff613
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1331702
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Use new APIs from <nvgpu/list.h> to access free
channel list
Define channel_gk20a_from_free_chs() to convert
a list node to struct channel_gk20a
Jira NVGPU-13
Change-Id: Idaf58f04be1c7fc553bea7c8de45951bf82bb340
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1303025
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Move code that touches host registers and instance block to fifo HAL.
This involves adding HAL ops for the fifo HAL functions that get
called from outside fifo. This clears responsibility of channel by
leaving it only managing channels in software and push buffers.
channel had member ramfc defined, but it was not used, to remove it.
pbdma_acquire_val consisted both of channel logic and hardware
programming. The channel logic was moved to the caller and only
hardware programming was moved.
Change-Id: Id005787f6cc91276b767e8e86325caf966913de9
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1322423
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Move the name field from struct gpu_ops up to struct gk20a. The field
is not a function op, so it doesn't belong in gpu_ops.
Replace all uses of dev_name() with use of g->name when possible.
JIRA NVGPU-16
Change-Id: Ic6e99e39258cbf3bb7c806962cbbd7de5126688f
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1328534
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
fifo reset_enable_hw is reorged to clear and enable pbdma/fifo interrupts
after all the required configuration such as configuring timeouts,
enabling timeout detections are taken care of.
JIRA GPUT19X-74
JIRA GPUT19X-47
Change-Id: Id780cc11d858db18f8d748c037954ede73298506
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: http://git-master/r/1325351
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Clean up gk20a.c by removing direct accesses to MC and moving the
accesses to happen via MC HAL.
The chip detection logic has to violate the HAL and call gk20a
version directly, because HAL ops cannot be set up before chip has
been identified.
Change-Id: I4cdd0ef3fcf7d3b561a3fca4247a8356fe8d18e1
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1321576
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
The fifo interrupt path was reading the PBDMA interrupt status
after clearing interrupts and this could lead to a situation in
which the host may have advanced to another channel, leading to
the recovery code resetting the wrong channel.
Bug 200278729
JIRA: EVLR-1036
Change-Id: I392423d1eaa8d23acf88454bf113c015e649e13d
Signed-off-by: David Nieto <dmartineznie@nvidia.com>
Reviewed-on: http://git-master/r/1326461
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
-gk20a_fifo_set_runlist_state() can be used to enable/disable runlists
scheduler. This change would be needed for t19x fifo recovery too
-Also delete gk20a_fifo_disable_all_engine_activity function as it is not
used anywhere.
JIRA GPUT19X-7
Change-Id: I6bb9a7574a473327f0e47060f32d52cd90551c6d
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: http://git-master/r/1315180
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Currently NVGPU_CHANNEL_FIFO_ERROR_MMU_ERR_FLT is being set in
error notifier for non mmu fault too. For fake mmu faults i.e.
trigger mmu fault cases, make sure proper notifiers are set and
driver is not depending on sending mmu error fault notifier.
This change is needed for t19x fifo recovery too.
NVGPU_CHANNEL_GR_ERROR_SW_METHOD (12), NVGPU_CHANNEL_GR_EXCEPTION(13)
and NVGPU_CHANNEL_FECS_ERR_UNIMP_FIRMWARE_METHOD (37) are new error
notifiers.
JIRA GPUT19X-7
Change-Id: Idee83e842c835bdba9eb18578aad0c372ea74c5d
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: http://git-master/r/1310563
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Init device_fatal, channel_fatal and restartable fifo intr pbdma s/w
variables for pbdma_intr_0 interrupt masks for each GPU version separately
pbdma_intr_0 field differences for each GPU version:-
-gk20a : bit 28 does not exists in hw
-gm20b : bit 8(lbreq), 20(xbarconnect) and 28 do not exist in hw
-gp10b : bit 8(lbreq), 20(xbarconnect) do not exist in hw. bit 28,
(syncpoint_illegal) added in hw but is not being handled.
-gk20a/gm20b/gp10b
bit 24 eng_reset and bit 25 semaphore always existed in hw but never
handled
JIRA GPUT19X-47
Change-Id: I209191f57c5ea5b15081b7dc2411801d3537017c
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: http://git-master/r/1325402
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Add CONFIG_TEGRA_GK20A_NVHOST and remove the TEGRA_GRHOST ||
TEGRA_HOST1X dependency in CONFIG_TEGRA_GK20A to allow using the iGPU
without the nvhost driver. Use the new config to guard syncpt-related
code.
Also make TEGRA_ACR depend on GK20A too so that it aligns properly under
gk20a in menuconfig.
Bug 1853519
Change-Id: I9e9b0a7915d000aae7930821627b7a01d08d3f5c
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1321303
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Use the new kmem API functions in the channel and channel
related code.
Also delete the usage of kasprintf() since that must be paired
with a kfree(). Since the kasprintf() doesn't use the nvgpu kmem
machinery (and is Linux specific) instead use a small buffer
statically allocated on the stack.
Bug 1799159
Bug 1823380
Change-Id: Ied0183f57372632264e55608f56539861cc0f24f
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1318312
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
When we get a PBDMA MMU fault, we won't be able to map the MMU id
into an engine id for reset. We still pass FIFO_INVAL_ENGINE_ID to
gk20a_fifo_should_defer_engine_reset() which causes an unnecessary
debug spew.
Check for FIFO_INVAL_ENGINE before calling
gk20a_fifo_should_defer_engine_reset().
Change-Id: I6f4a49be194cbc6070c1a1c667059de2ea79790f
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1321492
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Move all programming of FB to fb_*.c files, and remove the inclusion
of FB hardware headers from other files.
TLB invalidate function took previously a pointer to VM, but the new
API takes only a PDB mem_desc, because FB does not need to know about
higher level VM.
GPC MMU is programmed from the same function as FB MMU, so added
dependency to GR hardware header to FB.
GP106 ACR was also triggering a VPR fetch, but that's not applicable
to dGPU, so removed that call.
Change-Id: I4eb69377ac3745da205907626cf60948b7c5392a
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1321516
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
JIRA: EVLR-1004
(*) Refactor the non-stalling interrupt path to execute clear on the
top half, so on dGPU case processing of stalling interrupts does not
block non-stalling one.
(*) Use a worker thread to do semaphore wakeups and allow batching of
the non-stalling operations.
(*) Fix a bug where some gpus will not properly track the completion
of interrupts, preventing safe driver unloads
Change-Id: Icc90a3acba544c97ec6a9285ab235d337ab9eefa
Signed-off-by: David Nieto <dmartineznie@nvidia.com>
Reviewed-on: http://git-master/r/1312796
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Lakshmanan M <lm@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: Navneet Kumar <navneetk@nvidia.com>
Add a debugfs interface to profile the kickoff ioctl
it provides the probability distribution and separates the information
between time spent in: the full ioctl, the kickoff function, the amount
of time spent in job tracking and the amount of time doing pushbuffer
copies
JIRA: EVLR-1003
Change-Id: I9888b114c3fbced61b1cf134c79f7a8afce15f56
Signed-off-by: David Nieto <dmartineznie@nvidia.com>
Reviewed-on: http://git-master/r/1308997
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Implement a worker thread to replace the delayed works in channel
watchdog and job cleanups. Watchdog runs by polling the channel states
periodically, and job cleanup is performed on channels that are appended
on a work queue consumed by the worker thread. Handling both of these
two in the same thread makes it impossible for them to cause a deadlock,
as has previously happened.
The watchdog takes references to channels during checking and possibly
recovering channels. Jobs in the cleanup queue have an additional
reference taken which is released after the channel is processed. The
worker is woken up from periodic sleep when channels are added to the
queue.
Currently, the queue is only used for job cleanups, but it is extendable
for other per-channel works too. The worker can also process other
periodic actions dependent on channels.
Neither the semantics of timeout handling or of job cleanups are yet
significantly changed - this patch only serializes them into one
background thread.
Each job that needs cleanup is tracked and holds a reference to its
channel and a power reference, and timeouts can only be processed on
channels that are tracked, so the thread will always be idle if the
system is going to be suspended, so there is currently no need to
explicitly suspend or stop it.
Bug 1848834
Bug 1851689
Bug 1814773
Bug 200270332
Jira NVGPU-21
Change-Id: I355101802f50841ea9bd8042a017f91c931d2dc7
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1297183
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
fifo_pbdma_status__size_1_v() and fifo_engine_status__size_1_v()
are not same for all gpus. Use litter value to calculate chip
specific fifo*status__size_1(v)
JIRA GV11B-45
Change-Id: I3d3d45bf79d15e14739fcc18cb1ca987669d5c11
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: http://git-master/r/1312688
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Instead of using Linux APIs for mutex and spinlocks
directly, use new APIs defined in <nvgpu/lock.h>
Replace Linux specific mutex/spinlock declaration,
init, lock, unlock APIs with new APIs
e.g
struct mutex is replaced by struct nvgpu_mutex and
mutex_lock() is replaced by nvgpu_mutex_acquire()
And also include <nvgpu/lock.h> instead of including
<linux/mutex.h> and <linux/spinlock.h>
Add explicit nvgpu/lock.h includes to below
files to fix complilation failures.
gk20a/platform_gk20a.h
include/nvgpu/allocator.h
Jira NVGPU-13
Change-Id: I81a05d21ecdbd90c2076a9f0aefd0e40b215bd33
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1293187
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
mutex_is_locked() API is defined on Linux only
and not on other OS like QNX.
Hence remove use of this API for OS abstraction
support to nvgpu.
Instead of using mutex_is_locked(), use
mutex_trylock() for same purpose
Jira NVGPU-13
Change-Id: I542daf20a2294153da8e8bfe89e0dc0387297523
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1297184
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Move semaphore_gk20a.c drivers/gpu/nvgpu/common/ since the semaphore
code is common to all chips.
Move the semaphore_gk20a.h header file to drivers/gpu/nvgpu/include/nvgpu
and rename it to semaphore.h. Also update all places where the header
is inluced to use the new path.
This revealed an odd location for the enum gk20a_mem_rw_flag. This should
be in the mm headers. As a result many places that did not need anything
semaphore related had to include the semaphore header file. Fixing this
oddity allowed the semaphore include to be removed from many C files that
did not need it.
Bug 1799159
Change-Id: Ie017219acf34c4c481747323b9f3ac33e76e064c
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1284627
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
