Commit Graph

330 Commits

Author SHA1 Message Date
Terje Bergstrom
3ba374a5d9 gpu: nvgpu: gk20a: Use new error macro
gk20a_err() and gk20a_warn() require a struct device pointer,
which is not portable across operating systems. The new nvgpu_err()
and nvgpu_warn() macros take struct gk20a pointer. Convert code
to use the more portable macros.

JIRA NVGPU-16

Change-Id: Ia51f36d94c5ce57a5a0ab83b3c83a6bce09e2d5c
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1331694
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
GVS: Gerrit_Virtual_Submit
2017-04-10 19:04:19 -07:00
Deepak Nibade
db2ee5c127 gpu: nvgpu: use nvgpu list for buffer states
Use nvgpu list APIs instead of linux list APIs
for to store buffer states in gk20a_dmabuf_priv.

Jira NVGPU-13

Change-Id: I9666b2435804b132bb86bb74c0b20590749b153f
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1454689
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
2017-04-09 23:54:26 -07:00
Alex Waterman
50667e097b gpu: nvgpu: Rename nvgpu DMA APIs
Rename the nvgpu DMA APIs from gk20a_gmmu_alloc* to nvgpu_dma_alloc*.
This better reflects the purpose of the APIs (to allocate DMA suitable
memory) and avoids confusion with GMMU related code.

JIRA NVGPU-12

Change-Id: I673d607db56dd6e44f02008dc7b5293209ef67bf
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1325548
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-04-06 18:15:04 -07:00
Alex Waterman
8f2d4a3f4a gpu: nvgpu: Move DMA API to dma.h
Make an nvgpu DMA API include file so that the intricacies of the
Linux DMA API can be hidden from the calling code.

Also document the nvgpu DMA API.

JIRA NVGPU-12

Change-Id: I7578e4c726ad46344b7921179d95861858e9a27e
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1323326
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-04-06 18:14:58 -07:00
Alex Waterman
c9665079d7 gpu: nvgpu: rename mem_desc to nvgpu_mem
Renaming was done with the following command:

  $ find -type f | \
    xargs sed -i 's/struct mem_desc/struct nvgpu_mem/g'

Also rename mem_desc.[ch] to nvgpu_mem.[ch].

JIRA NVGPU-12

Change-Id: I69395758c22a56aa01e3dffbcded70a729bf559a
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1325547
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-04-06 18:14:53 -07:00
Alex Waterman
b69020bff5 gpu: nvgpu: Rename gk20a_mem_* functions
Rename the functions used for mem_desc access to nvgpu_mem_*.

JIRA NVGPU-12

Change-Id: Ibfdc1112d43f0a125e4487c250e3f977ffd2cd75
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1323325
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-04-06 18:14:48 -07:00
Alex Waterman
fa4ecf5730 gpu: nvgpu: Split mem_desc from MM code
Split the mem_desc code out from the MM code. This is to help
simplify the MM code and make it easier to abstract the DMA
allocation routines.

JIRA NVGPU-12

Change-Id: I2ccb643efe6bbed80d1360a580ff5593acb407bd
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1323324
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-04-06 18:14:42 -07:00
Deepak Nibade
ce3c30f14f gpu: nvgpu: use nvgpu rbtree to store mapped buffers
Use nvgpu rbtree instead of linux rbtree to store
mapped buffers for each VM

Move to use "struct nvgpu_rbtree_node" instead of
"struct rb_node"
And similarly use rbtree APIs from <nvgpu/rbtree.h>
instead of linux APIs

Jira NVGPU-13

Change-Id: Id96ba76e20fa9ecad016cd5d5a6a7d40579a70f2
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1453043
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-04-06 10:57:28 -07:00
Alex Waterman
c86e940e11 gpu: nvgpu: Remove last Linux kmem usage
Replace the last of the Linux kmem API usage with nvgpu kmem
calls instead. Several places are left alone - allocating the
struct gk20a in particular.

Also one function was updated in the clk code to take a struct
gk20a as an argument so that it could use nvgpu_kmalloc().

Bug 1799159
Bug 1823380

Change-Id: I84fc3f8e19c63d6265bac6098dc727d93e3ff613
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1331702
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-04-04 16:57:03 -07:00
Konsta Holtta
2119213467 gpu: nvgpu: use NO_KERNEL_MAPPING for default vidmem
Commit 8f3875393e ("abstract away dma
alloc attrs") added an implicit NVGPU_DMA_NO_KERNEL_MAPPING for the
explicit vidmem buffer allocation path. The default allocation api that
chooses vidmem or sysmem based on available support still passed a zero
flag and produced a warning when the flag should have been there. Force
the NO_KERNEL_MAPPING flag on currently via the default-allocator api.

Commit 8a15e02ca9 ("gpu: nvgpu: add
NO_KERNEL_MAPPING for alloc_map_vid") did the same for flagless
alloc-and-map calls but this default alloc-only call got overlooked.

Also, store the flags in the mem_desc during allocation, just like in
the sysmem allocator, to be checked during freeing.

Bug 1896734
Bug 1853519

Change-Id: I4b4182b4fd52298bdd2a77197c095a23d2a67c4a
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1331252
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-04-03 14:04:04 -07:00
Deepak Nibade
cd3cf04cac gpu: nvgpu: use nvgpu list for VA lists
Use nvgpu list APIs instead of linux list APIs
for reserved VA list and buffer VA list

Jira NVGPU-13

Change-Id: I83c02345d54bca03b00270563567227510cfce6b
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1454013
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-04-03 08:55:20 -07:00
Deepak Nibade
371b173c12 gpu: nvgpu: use nvgpu list for vidmem clear list
Use nvgpu list APIs instead of linux list APIs
for vidmem clear list

Jira NVGPU-13

Change-Id: I13f7c5a54fb199d15ad1402216f3275f0f0474af
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1454012
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-04-03 08:55:20 -07:00
Deepak Nibade
0d8830394a gpu: nvgpu: use nvgpu list for page chunks
Use nvgpu list APIs instead of linux list APIs
to store chunks of page allocator

Jira NVGPU-13

Change-Id: I63375fc2df683e018c48a90b76eca368438cc32f
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1326814
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-04-03 08:55:19 -07:00
Alex Waterman
dd88aed5cc gpu: nvgpu: Split out pramin code
Split out the pramin interface code in preparation for splitting
out the mem_desc code.

JIRA NVGPU-12

Change-Id: I3f03447ea213cc15669b0934fa706e7cb22599b7
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1323323
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-03-31 17:21:34 -07:00
Alex Waterman
7010bf8839 gpu: nvgpu: Use new kmem API functions (gk20a mm)
Use the new kmem API functions in gk20a's mm code. Add a struct
gk20a pointer to the dmabuf priv struct so that the cleanup
function has access to the gk20a struct. Also add a gk20a pointer
to some of the sg table functions so that they can use the nvgpu
kmem APIs.

Bug 1799159
Bug 1823380

Change-Id: I85a307c6bf862627c5b1af0e077283b48d78fa72
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1318321
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-03-30 12:36:04 -07:00
Terje Bergstrom
3e39798997 gpu: nvgpu: Remove unnecessary use of dev_name()
Move the name field from struct gpu_ops up to struct gk20a. The field
is not a function op, so it doesn't belong in gpu_ops.

Replace all uses of dev_name() with use of g->name when possible.

JIRA NVGPU-16

Change-Id: Ic6e99e39258cbf3bb7c806962cbbd7de5126688f
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1328534
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-03-28 12:05:03 -07:00
Konsta Holtta
8a15e02ca9 gpu: nvgpu: add NO_KERNEL_MAPPING for alloc_map_vid
Commit 8f3875393e ("abstract away dma
alloc attrs") added an implicit NVGPU_DMA_NO_KERNEL_MAPPING for the
function that allocates vidmem buffers, but the functions
gk20a_gmmu_alloc_map_vid and gk20a_gmmu_alloc_map got overlooked and use
no flags which now triggers a warning.  Make those cases alloc the
vidmem buffer with the no kernel mapping flag as it should have been,
because kernel mappings of vidmem are not supported.

Bug 1853519

Change-Id: I9f29c9d310f97c9bd5f279674decf61efb3e75ea
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1328953
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-03-27 13:56:38 -07:00
Terje Bergstrom
b45a67934f gpu: nvgpu: Use nvgpu_timeout for all loops
There were still a few remaining loops where we did not use
nvgpu_timeout and required Tegra specific functions for detecting if
timeout should be skipped. Replace all of them with nvgpu_timeout and
remove including chip-id.h where possible.

FE power mode timeout loop also used wrong delay value. It always
waited for the whole max timeout instead of looping with smaller
increments.

If SEC2 ACR boot fails to halt, we should not try to check ACR result
from mailbox. Add an early return for that case.

JIRA NVGPU-16

Change-Id: I9f0984250d7d01785755338e39822e6631dcaa5a
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1323227
2017-03-27 10:48:31 -07:00
David Nieto
e0f2afe5eb gpu: nvgpu: refactor teardown to support unbind
This change refactors the teardown in remove to ensure that it is
possible to unload the driver while leaving fds open. This is achieved
by making sure that the SW state is kept alive till all fds are closed
and by checking that subsequent calls to ioctls after the teardown fail.

Normally, this would be achieved ny calls into gk20a_busy(), but in
kickoff we dont call into that to reduce latency, so we need to check
the driver status directly, and also in some of the functions
as we need to make sure the ioctl does not dereference the device or
platform struct

bug 200277762
JIRA: EVLR-1023

Change-Id: I163e47a08c29d4d5b3ab79f0eb531ef234f40bde
Signed-off-by: David Nieto <dmartineznie@nvidia.com>
Reviewed-on: http://git-master/r/1320219
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Shreshtha Sahu <ssahu@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
2017-03-25 02:06:55 -07:00
Terje Bergstrom
81660ab58c gpu: nvgpu: Enable CE always
All GPUs have a copy engine. So delete the flag has_ce, because
it's always true.

JIRA NVGPU-16

Change-Id: I89db74c7cf66b24db84301b79832862ef28100b9
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1325355
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-03-24 09:08:23 -07:00
Terje Bergstrom
4492c62ffe gpu: nvgpu: Add bus HAL
Add bus HAL and move all bus related hardware sequencing to that file:
BAR1 binding, timer access, and interrupt handling.

Change-Id: Ibc5f5797dc338de10749b446a7bdbcae600fecb4
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1323353
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-03-23 08:44:07 -07:00
Seema Khowala
971a3c8050 gpu: nvgpu: add mm ops for mmu_fault_pending
This change is needed for t19x mmu fault handling.

Change-Id: I7f9190ab305f699401f6b0033b6a93dd8b4fc3cd
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: http://git-master/r/1315201
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-03-22 10:11:21 -07:00
Konsta Holtta
fcd7fce9bc gpu: nvgpu: use dma-attr wrappers for K4.9 compatibility
On kernel 4.9, the DMA API has changed, so use the NVIDIA compatibility
wrappers from dma-attrs.h to allow the code to build for both 4.4 and
4.9.

Bug 1853519

Change-Id: I0196936e81c7f72b41b38a67f42af0dc0b5518df
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1321102
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-03-21 15:01:48 -07:00
Konsta Holtta
8f3875393e gpu: nvgpu: abstract away dma alloc attrs
Don't use enum dma_attr in the gk20a_gmmu_alloc_attr* functions, but
define nvgpu-internal flags for no kernel mapping, force contiguous, and
read only modes. Store the flags in the allocated struct mem_desc and
only use gk20a_gmmu_free, remove gk20a_gmmu_free_attr. This helps in OS
abstraction. Rename the notion of attr to flags.

Add implicit NVGPU_DMA_NO_KERNEL_MAPPING to all vidmem buffers
allocated via gk20a_gmmu_alloc_vid for consistency.

Fix a bug in gk20a_gmmu_alloc_map_attr that dropped the attr
parameter accidentally.

Bug 1853519

Change-Id: I1ff67dff9fc425457ae445ce4976a780eb4dcc9f
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1321101
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-03-21 15:01:47 -07:00
Terje Bergstrom
ca762e4220 gpu: nvgpu: Move all FB programming to FB HAL
Move all programming of FB to fb_*.c files, and remove the inclusion
of FB hardware headers from other files.

TLB invalidate function took previously a pointer to VM, but the new
API takes only a PDB mem_desc, because FB does not need to know about
higher level VM.

GPC MMU is programmed from the same function as FB MMU, so added
dependency to GR hardware header to FB.

GP106 ACR was also triggering a VPR fetch, but that's not applicable
to dGPU, so removed that call.

Change-Id: I4eb69377ac3745da205907626cf60948b7c5392a
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1321516
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-03-17 08:44:03 -07:00
Konsta Holtta
7fd72e3135 gpu: nvgpu: cancel vidmem worker only if supported
Cancel the vidmem.clear_mem_worker during suspend only if vidmem is
enabled via kernel config. Otherwise it's not initialized.

Bug 1853519

Change-Id: If88c756ae14f348eddda01218fa218480217388c
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1321118
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-by: David Martinez Nieto <dmartineznie@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-03-16 09:17:22 -07:00
Terje Bergstrom
85cb10c313 gpu: nvgpu: Remove unused function gk20a_get_phys_from_iova
Remove unused function gk20a_get_phys_from_iova. At the same time
remove the #include for iommu.h, which was only needed by this
function.

Change-Id: Ia858b0ad5fe7e423d650aa9f82e430f419f2a492
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1319070
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-03-14 11:47:17 -07:00
Terje Bergstrom
503a5e6826 gpu: nvgpu: Do not use pm_runtime calls directly
mm_gk20a.c had direct calls to pm_runtime_put_noidle(). Replace
them with calls to wrapper gk20a_idle_nosuspend() to prevent
unnecessary dependencies to Linux.

Change-Id: Iaf8b9255750be2f3e1aa39587c1a4a3cbeacc67f
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1319069
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-03-14 11:47:11 -07:00
Deepak Nibade
9efadcdfc0 gpu: nvgpu: check return value of mutex_init for semaphores
- check return value of nvgpu_mutex_init for semaphores
- add corresponding nvgpu_mutex_destroy calls

Jira NVGPU-13

Change-Id: I5404dbd29e3fce29f1a445eb2e6ce8e1d1b616c4
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1317138
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: Navneet Kumar <navneetk@nvidia.com>
2017-03-14 11:46:53 -07:00
Alex Waterman
707ea45e0f gpu: nvgpu: kmem abstraction and tracking
Implement kmem abstraction and tracking in nvgpu. The abstraction
helps move nvgpu's core code away from being Linux dependent and
allows kmem allocation tracking to be done for Linux and any other
OS supported by nvgpu.

Bug 1799159
Bug 1823380

Change-Id: Ieaae4ca1bbd1d4db4a1546616ab8b9fc53a4079d
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1283828
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-03-03 10:34:48 -08:00
Alex Waterman
3966efc2e5 gpu: nvgpu: Give nvgpu_kalloc a less generic name
Change nvgpu_kalloc() to nvgpu_big_[mz]alloc(). This is necessary
since the natural free function name for this is nvgpu_kfree() but
that conflicts with nvgpu_k[mz]alloc() (implemented in a subsequent
patch).

This API exists becasue not all allocation sizes can be determined
at compile time and in some cases sizes may vary across the system
page size. Thus always using kmalloc() could lead to OOM errors due
to fragmentation. But always using vmalloc() is wastful of memory
for small allocations. This API tries to alleviate those problems.

Bug 1799159
Bug 1823380

Change-Id: I49ec5292ce13bcdecf112afbb4a0cfffeeb5ecfc
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1283827
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-03-03 10:34:43 -08:00
Deepak Nibade
64e1782aee gpu: nvgpu: optimize duplicate buffer lookup in case of fixed offsets
In gk20a_vm_map_duplicate_locked(), we always do a linear search
in rb-tree to find a duplicate entry of the buffer

In case NVGPU_AS_MAP_BUFFER_FLAGS_FIXED_OFFSET is set, we first
traverse whole rb-tree linearly and then compare offset_align
with the address searched from rb-tree

If size of rb-tree is very large this linear lookup takes upto
7mS and causes huge delays

Hence in case of NVGPU_AS_MAP_BUFFER_FLAGS_FIXED_OFFSET, we can
use offset_align to perform a binary search on rb-tree and then
verify that dmabuf and kind match with the node obtained from
the search
This saves a lot of time per-lookup

Bug 1874516

Change-Id: Ia4924b64d66e586c14341ae2e2283beac394bf6f
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1309343
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-03-02 07:53:55 -08:00
Deepak Nibade
8cdb91c527 gpu: nvgpu: remove use of DEFINE_MUTEX()
API DEFINE_MUTEX() is defined in Linux and might
not be available in other OSs.
Hence remove its usage from nvgpu

Declare and explicitly initialize below mutexes
for both nvgpu and vgpu
g->mm.priv_lock
g->mm.tlb_lock

Jira NVGPU-13

Change-Id: If72885a6da0227a1552303206172f1f2b751471d
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1298042
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-02-22 04:15:08 -08:00
Deepak Nibade
8ee3aa4b31 gpu: nvgpu: use common nvgpu mutex/spinlock APIs
Instead of using Linux APIs for mutex and spinlocks
directly, use new APIs defined in <nvgpu/lock.h>

Replace Linux specific mutex/spinlock declaration,
init, lock, unlock APIs with new APIs
e.g
struct mutex is replaced by struct nvgpu_mutex and
mutex_lock() is replaced by nvgpu_mutex_acquire()

And also include <nvgpu/lock.h> instead of including
<linux/mutex.h> and <linux/spinlock.h>

Add explicit nvgpu/lock.h includes to below
files to fix complilation failures.
gk20a/platform_gk20a.h
include/nvgpu/allocator.h

Jira NVGPU-13

Change-Id: I81a05d21ecdbd90c2076a9f0aefd0e40b215bd33
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1293187
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-02-22 04:15:02 -08:00
Alex Waterman
e7a0c0ae8b gpu: nvgpu: Move from gk20a_ to nvgpu_ in semaphore code
Change the prefix in the semaphore code to 'nvgpu_' since this code
is global to all chips.

Bug 1799159

Change-Id: Ic1f3e13428882019e5d1f547acfe95271cc10da5
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1284628
Reviewed-by: Varun Colbert <vcolbert@nvidia.com>
Tested-by: Varun Colbert <vcolbert@nvidia.com>
2017-02-13 18:15:03 -08:00
Alex Waterman
aa36d3786a gpu: nvgpu: Organize semaphore_gk20a.[ch]
Move semaphore_gk20a.c drivers/gpu/nvgpu/common/ since the semaphore
code is common to all chips.

Move the semaphore_gk20a.h header file to drivers/gpu/nvgpu/include/nvgpu
and rename it to semaphore.h. Also update all places where the header
is inluced to use the new path.

This revealed an odd location for the enum gk20a_mem_rw_flag. This should
be in the mm headers. As a result many places that did not need anything
semaphore related had to include the semaphore header file. Fixing this
oddity allowed the semaphore include to be removed from many C files that
did not need it.

Bug 1799159

Change-Id: Ie017219acf34c4c481747323b9f3ac33e76e064c
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1284627
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-02-13 18:14:45 -08:00
Alex Waterman
7e403974d3 gpu: nvgpu: Simplify ref-counting on VMs
Simplify ref-counting on VMs: take a ref when a VM is bound to a
channel and drop a ref when a channel is freed.

Previously ref-counts were scattered over the driver. Also the CE
and CDE code would bind channels with custom rolled code. This was
because the gk20a_vm_bind_channel() function took an as_share as
the VM argument (the VM was then inferred from that as_share).
However, it is trivial to abtract that bit out and allow a central
bind channel function that just takes a VM and a channel.

Bug 1846718

Change-Id: I156aab259f6c7a2fa338408c6c4a3a464cd44a0c
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1261886
Reviewed-by: Richard Zhao <rizhao@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-02-07 14:54:02 -08:00
Alex Waterman
95a3eb454c gpu: nvgpu: Conditional address space unification
Allow platforms to choose whether or not to have unified GPU
VA spaces. This is useful for the dGPU where having a unified
address space has no problems. On iGPUs testing issues is
getting in the way of enabling this feature.

Bug 1396644
Bug 1729947

Change-Id: I65985f1f9a818f4b06219715cc09619911e4824b
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1265303
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-01-31 16:23:07 -08:00
Alex Waterman
b9b94c073c gpu: nvgpu: Remove separate fixed address VMA
Remove the special VMA that could be used for allocating fixed
addresses. This feature was never used and is not worth maintaining.

Bug 1396644
Bug 1729947

Change-Id: I06f92caa01623535516935acc03ce38dbdb0e318
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1265302
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-01-31 16:23:07 -08:00
Alex Waterman
321537b8ed gpu: nvgpu: Cleanup gk20a_init_vm()
Cleanup and simplify the gk20a_init_vm() function to ease the
implementation of a platform dependent address space unification
decision.

Bug 1396644
Bug 1729947

Change-Id: Id8487d0e3d3c65e3357e3528063fb17c8a85f7da
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1265301
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-01-31 16:23:07 -08:00
Alex Waterman
d630f1d99f gpu: nvgpu: Unify the small and large page address spaces
The basic structure of this patch is to make the small page allocator
and the large page allocator into pointers (where they used to be just
structs). Then assign each of those pointers to the same actual
allocator since the buddy allocator has supported mixed page sizes
since its inception.

For the rest of the driver some changes had to be made in order to
actually support mixed pages in a single address space.

1. Unifying the allocation page size determination

   Since the allocation and map operations happen at distinct
   times both mapping and allocation of GVA space must agree
   on page size. This is because the allocation has to separate
   allocations into separate PDEs to avoid the necessity of
   supporting mixed PDEs.

   To this end a function __get_pte_size() was introduced which
   is used both by the balloc code and the core GPU MM code. It
   determines page size based only on the length of the mapping/
   allocation.

2. Fixed address allocation + page size

   Similar to regular mappings/GVA allocations fixed address
   mapping page size determination had to be modified. In the
   past the address of the mapping determined page size since
   the address space split was by address (low addresses were
   small pages, high addresses large pages). Since that is no
   longer the case the page size field in the reserve memory
   ioctl is now honored by the mapping code. When, for instance,
   CUDA makes a memory reservation it specifies small or large
   pages. When CUDA requests mappings to be made within that
   address range the page size is then looked up in the reserved
   memory struct.

   Fixed address reservations were also modified to now always
   allocate at a PDE granularity (64M or 128M depending on
   large page size. This prevents non-fixed allocations from
   ending up in the same PDE and causing kernel panics or GMMU
   faults.

3. The rest...

   The rest of the changes are just by products of the above.
   Lots of places required minor updates to use a pointer to
   the GVA allocator struct instead of the struct itself.

Lastly, this change is not truly complete. More work remains to be
done in order to fully remove the notion that there was such a thing
as separate address spaces for different page sizes. Basically after
this patch what remains is cleanup and proper documentation.

Bug 1396644
Bug 1729947

Change-Id: If51ab396a37ba16c69e434adb47edeef083dce57
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1265300
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-01-31 16:23:07 -08:00
Alex Waterman
793791ebb7 gpu: nvgpu: use map_offset for PTE size computation
Make sure that map_offset is set to the fixed map address or 0)
before determining PTE size. Then use map_offset instead of
offset_align for computing the PTE size since offset_align
could be either an alignment ora fixed mapping offset.

Also is the minimum of the buffer size and the buffer alignment
for computing page size. This is necessary is the GMMU is doing
page gathering (i.e the buffer does not appear as a continguous
IOMMU range to the GPU). Is such cases a large page sized buffer
may be made up of a bunch of discontiguous 4k pages.

Bug 1396644
Bug 1729947

Change-Id: I6464ee6a4ccab2495ccb31cd1ddf1db467d2b215
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1271359
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-01-31 16:23:07 -08:00
Shardar Shariff Md
a470647ad7 gpu: nvgpu: use soc/tegra/chip-id.h for soc header
The soc tegra headers are unified and moved all the content of
linux/tegra-soc.h to the soc/tegra/chip-id.h to have the
single soc header for Tegra.

Change-Id: I281e19dd3eb1538b8dfbea4eb0779fb64d1fcffa
Signed-off-by: Shardar Shariff Md <smohammed@nvidia.com>
Reviewed-on: http://git-master/r/1288365
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-01-20 08:24:01 -08:00
Alex Waterman
6e2237ef62 gpu: nvgpu: Use timer API in gk20a code
Use the timers API in the gk20a code instead of Linux specific
API calls.

This also changes the behavior of several functions to wait for
the full timeout for each operation that can timeout. Previously
the timeout was shared across each operation.

Bug 1799159

Change-Id: I2bbed54630667b2b879b56a63a853266afc1e5d8
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1273826
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-01-18 16:46:33 -08:00
Alex Waterman
b928f10d37 gpu: nvgpu: Start re-organizing the HW headers
Reorganize the HW headers of gk20a. The headers are moved to a
new directory:

  include/nvgpu/hw/gk20a

And from the code are included like so:

  #include <nvgpu/hw/gk20a/hw_pwr_gk20a.h>

This is the first step in reorganizing all of the HW headers for
gm20b, gm206, etc. This is part of a larger effort to re-structure
and make the driver more readable and scalable.

Bug 1799159

Change-Id: Ic151155cbc2e6f75009f2d9d597b364a1bed2c4c
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1244790
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-01-11 12:44:14 -08:00
Alex Waterman
6df3992b60 gpu: nvgpu: Move allocators to common/mm/
Move the GPU allocators to common/mm/ since the allocators are common
code across all GPUs. Also rename the allocator code to move away from
gk20a_ prefixed structs and functions.

This caused one issue with the nvgpu_alloc() and nvgpu_free() functions.
There was a function for allocating either with kmalloc() or vmalloc()
depending on the size of the allocation. Those have now been renamed to
nvgpu_kalloc() and nvgpu_kfree().

Bug 1799159

Change-Id: Iddda92c013612bcb209847084ec85b8953002fa5
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1274400
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-01-09 12:33:16 -08:00
Alex Waterman
91d977ced4 gpu: nvgpu: Misc fixes for crashes on shutdown
Fix miscellaneous issues seen during driver shutdown.

 o  Make sure pointers are valid before accessing them.
 o  Busy the GPU during channel timeout.
 o  Cancel delayed work on channels.
 o  Avoid access to channels that may have been freed.

Bug 1816516
Bug 1807277

Change-Id: I62df40373fdfb1c4a011364e8c435176a08a7a96
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1250026
(cherry picked from commit 64a95fc96c8ef7c5af9c53c4bb3402626e0d2f60)
Reviewed-on: http://git-master/r/1274474
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-01-04 15:53:55 -08:00
Konsta Holtta
92fe000749 gpu: nvgpu: rename timeout_check to timeout_expired
Change "check" to "expired" in nvgpu_timeout_check* and append _expired
to nvgpu_timeout_peek to clarify what the boolean-like return value
means and thus avoid bugs.

Bug 200260715

Change-Id: I47e097ee922e856005a79fa9e27eddb1c8d77f8b
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1269366
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2016-12-19 15:40:36 -08:00
Terje Bergstrom
a4731b3282 gpu: nvgpu: Use end of vidmem as bootstrap region
Instead of hard coding bootstrap region, it should always be set to
the last 256MB of vidmem.

Bug 200244445

Change-Id: I91779d1bf861f4f23a0b646f70b1febbbc4581b5
Signed-off-by: David Nieto <dmartineznie@nvidia.com>
Reviewed-on: http://git-master/r/1242409
Reviewed-by: David Martinez Nieto <dmartineznie@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-on: http://git-master/r/1267124
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2016-12-09 15:05:12 -08:00
Konsta Holtta
54810444d2 gpu: nvgpu: fix timeout retry usage in mm_gk20a.c
Loop conditions of timeout checking introduced in commit
2109478311 ("gpu: nvgpu: Use timeout retry
API in mm_gk20a.c") were flipped by accident, so each usage in a loop
actually did not wait enough but ran only one iteration. Fix the
conditions to loop as long as the timeout is NOT expired.

Also restore l2 flush timeout to 10 ms from 1, which was done in commit
030ef82bdd ("gpu: nvgpu: increase l2 flush
timeout") but overwritten by the above "use timeout" commit.

Bug 200260715

Change-Id: I0db16be79a1a27caa3d97fac9d4361582cc232e8
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1268482
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2016-12-09 13:44:53 -08:00