Instead of comptag index we were dumping an offset in buffer.
Change-Id: Iaa07919c8d87009227556eacbcb6dcbd83954c7d
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/718597
Reviewed-by: Automatic_Commit_Validation_User
Reduce amount of duplicate code around memory allocation by using
common helpers, and common data structure for storing results of
allocations.
Bug 1605769
Change-Id: I7c1662b669ed8c86465254f6001e536141051ee5
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/720435
Reduce amount of duplicate code around memory allocation by
introducing a variant of allocation helper that does not map the
allocated buffer to kernel address space.
NO_KERNEL_MAPPING allocations return a struct page **, so store the
results of allocation in a new field of mem_desc.
Bug 1605769
Change-Id: Ib760b9e6d34b229b04d1fb4f3abf10648670fc69
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/721029
Introduce mem_desc, which holds all information needed for a buffer.
Implement helper functions for allocation and freeing that use this
data type.
Change-Id: I82c88595d058d4fb8c5c5fbf19d13269e48e422f
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/712699
Added support for dumping vpr/wpr info for gm20b.
This dump info called when ever gk20a_mm_fb_flush
is timed-out.
Bug 200082817
Change-Id: I21b0372d0e3f976a189c9c428c015165b715bf88
Signed-off-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: http://git-master/r/711439
(cherry picked from commit b69897d71c8f6119b49ceb8d3273cdb354178cc5)
Reviewed-on: http://git-master/r/712675
GVS: Gerrit_Virtual_Submit
Reviewed-by: Yu-Huan Hsu <yhsu@nvidia.com>
Fix below sparse warnings by making below APIs static :
kernel/drivers/gpu/nvgpu/gk20a/mm_gk20a.c:1795:5:
warning: symbol 'update_gmmu_pde_locked' was not declared. Should it be
static?
kernel/drivers/gpu/nvgpu/gk20a/mm_gk20a.c:1841:5:
warning: symbol 'update_gmmu_pte_locked' was not declared. Should it be
static?
Bug 200067946
Change-Id: I8158aaf503378b176cfd5cc129db9557803003c1
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/713024
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
don't unmap compbits_buf explicitly from system vm early but store it in
the dmabuf's private data, and unmap it later when all user mappings to
that buffer have been disappeared.
Bug 1546619
Change-Id: I333235a0ea74c48503608afac31f5e9f1eb4b99b
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/661949
(cherry picked from commit ed2177e25d9e5facfb38786b818330798a14b9bb)
Reviewed-on: http://git-master/r/661835
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
Pass always the directory structure to mm functions instead of
pointers to members to it. Also split update_gmmu_ptes_locked()
into smaller functions, and turn the hard
coded MMU levels (PDE, PTE) into run-time parameters.
Change-Id: I315ef7aebbea1e61156705361f2e2a63b5fb7bf1
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/672485
Reviewed-by: Automatic_Commit_Validation_User
Map/unmap buffers for HWPM and deal with its instance block, the minimum
work required to run the HWPM via regops from userspace.
Bug 1517458
Bug 1573150
Change-Id: If14086a88b54bf434843d7c2fee8a9113023a3b0
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/673689
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
We have infinite timeouts for loops in emulation. Some functions with
the loops still return error if we exceed the original retry count.
Change-Id: I1f9ddbfc0acd9f30f6bd49d9e748d8d8fbefa154
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/709491
Introduce a new struct gk20a_mm_entry. Allocate and store PDE and PTE
arrays using the same structure. Always pass pointer to this struct
when possible between functions in memory code.
Change-Id: Ia4a2a6abdac9ab7ba522dafbf73fc3a3d5355c5f
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/696414
Use busy looping on L2 and TLB maintenance operations. This speeds
them up by an order of magnitude.
Add also trace points to measure performance for memory ops and
interrupt processing.
Change-Id: Ic4a8525d3d946b2b8f57b4b8ddcfc61605619399
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/681640
Pass vm instead of as share to the userspace buffer mapping functions,
since they need to be called also from other places than just the AS
device ioctls, and as share is specific to them.
Bug 1573150
Change-Id: I994872f23ea7b1582361f3f4fabbd64b4786419c
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/674020
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
Compression page size varies depending on architecture. Make it
129kB on gk20a and gm20b.
Also export some common functions from gm20b.
Bug 1592495
Change-Id: Ifb1c5b15d25fa961dab097021080055fc385fecd
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/673790
Decrement the ref count on all mapped_buffers belonging to a va_node
when a va_node is freed. This prevents userspace from leaking some
mapped_buffers in some cases.
This does prevent userspace from keeping a buffer around after freeing
a space allocation if the buffer in question is not otherwise ref
counted. Not sure if this is a bad thing for userspace or not.
Bug 1600686
Change-Id: I659ccbda5935d44086fd367bd2110f7d0f066194
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/676629
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
Implement several fixes for allowing the GVA address space to grow
to larger than 32GB and increase the address space to 128GB.
o Implement dynamic allocation of PDE backing pages. The memory
to store the PDE entries was hard coded to 1 page. Now the
number of pages necessary is computed dynamically based on the
size of the address space and the size of large pages.
o Fix an arithmetic problem in the gm20b sparse texture code
that caused large address spaces to be truncated when sparse
PDEs/PTEs were being filled in. This caused a kernel panic
when freeing the address space since a lot of the backing
PTE memory was not allocated.
o Change the address space split for large and small pages. Small
pages now occupy the bottom 16GB of the address space. Large
pages are used for the rest of the address space. Now, with a
128GB address space, there are 112GB of large page GVA available.
This patch exists to allow large (16GB) sparse textures to be allocated
without running into lack of memory issues and kernel panics.
Bug 1574267
Change-Id: I7c59ee54bd573dfc53b58c346156df37a85dfc22
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/671204
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
Fix a memory leak: add the newly created state to the dmabuf priv's
state list, instead of the other way around.
Bug 1594784
Bug 200064154
Change-Id: I939746a254bb8bf4d06de7fcecba06c191da665f
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/668758
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Lauri Peltonen <lpeltonen@nvidia.com>
Reviewed-by: Arto Merilainen <amerilainen@nvidia.com>
Remove support for gk20a sparse textures. We're using implementation
from user space, so gk20a code is never invoked.
Also removes ref_cnt for PTEs, so we never free PTEs when unmapping
pages, but only at VM delete time.
Change-Id: I04d7d43d9bff23ee46fd0570ad189faece35dd14
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/663294
Make mem_desc a generic container for buffers. Add functions for
allocating and mapping buffers to an address space which store their
data in mem_desc.
Change-Id: I031643442c6fd41f5e7222fe9b7bfcaf9b784db5
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/660908
GVS: Gerrit_Virtual_Submit
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
A device can be iommu'able whenever it's registered so that this patch
detects its iommu'ability dynamically.
Bug 1577389
Change-Id: I8ea20e5dd997fc1a399f517c17783323f238ecc3
Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>
Reviewed-on: http://git-master/r/606019
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Retrieve number of physical page bits based on chip.
Bug 1567274
Change-Id: I5a0f6a66be37f2cf720d66b5bdb2b704cd992234
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/601700
When doing l2 invalidate or l2 flush, first check if the hw is powered
on. If it is not, nothing is done, as there are no hardware registers
available. As a side-effect, this may race so that the hardware stays
unrailgated.
Change-Id: I8bdbfcee3545355435d4ae01476188eb1b8b8817
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/594441
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
Add sanity check for big_page_size parameter to avoid invoking
gk20a_init_vm() with a bogus big page size, potentially hitting a
BUG_ON there. Also, reorganize the code a bit to avoid memory leak in
case of bogus big page size, and properly forward the return value
from gk20a_init_vm().
Change-Id: I4eeada75415d2e9539b5e8859099cce35cd86db3
Signed-off-by: Sami Kiminki <skiminki@nvidia.com>
Reviewed-on: http://git-master/r/594469
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
gk20a_vm_alloc_share() fails when the default big page size is
requested but ops.mm.set_big_page_size is unset. Rework the logic a
bit to allow userspace to explicitly request the default big page
size, too.
Change-Id: I2a28c6d979fbf1dde5559ce9eb5f1310d232e27f
Signed-off-by: Sami Kiminki <skiminki@nvidia.com>
Reviewed-on: http://git-master/r/590456
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
Comptags are assigned per 128k. For 64k pages this means we need to
assign same index to two pages. Change the logic to use byte sizes.
Change-Id: If298d6b10f1c1cad8cd390f686d22db103b02d12
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/594403
When using CDE firmware v1, combine H and V swizzling passes into one
pushbuffer submission. This removes one GPU context switch, almost
halving the time taken for swizzling.
Map only the compbit part of the destination surface.
Bug 1546619
Change-Id: I95ed4e4c2eefd6d24a58854d31929cdb91ff556b
Signed-off-by: Jussi Rasanen <jrasanen@nvidia.com>
Reviewed-on: http://git-master/r/553234
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
Trying to kfree pmu.vm or bar1.vm is not allowed, since they are not
directly allocated. Separate the vm kfree from the actual vm support
removal, so that they can be done individually.
Change-Id: I7628f546b94e0de909371ce315e4cb065e5ef953
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/592112
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Some vm's do not have big pages.
Bug 1476801
Change-Id: Ic82ca7a1380834ea30582631af224c81fd01e4bb
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/592113
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Fix below sparse warnings :
warning: Using plain integer as NULL pointer
warning: symbol <variable/funcion> was not declared. Should it be static?
warning: Initializer entry defined twice
Also, remove dead functions
Bug 1573254
Change-Id: I29d71ecc01c841233cf6b26c9088ca8874773469
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/593363
Reviewed-by: Amit Sharma (SW-TEGRA) <amisharma@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Sachin Nikam <snikam@nvidia.com>
Allow compression always when page size matches the big page
size for the context.
Bug 1558739
Change-Id: I27b0aed06c24d69bd1555626b9affb1149536615
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/590422
Convert GR and LTC HALs to use const structs, and initialize them
with macros.
Bug 1567274
Change-Id: Ia3f24a5eccb27578d9cba69755f636818d11275c
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/590371
VM size should not depend on CPU architecture. It should be always
u64.
Change-Id: I81539807f6674877fd04f0079b2bec05b2a0640d
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/562466
Implement support for 64kB large page size. Add an API to create an
address space via IOCTL so that we can accept flags, and assign one
flag for enabling 64kB large page size.
Also adds APIs to set per-context large page size. This is possible
only on Maxwell, so return error if caller tries to set large page
size on Kepler.
Default large page size is still 128kB.
Change-Id: I20b51c8f6d4a984acae8411ace3de9000c78e82f
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Merge initialization code from gk20a_init_system_vm(),
gk20a_init_bar1_vm() and gk20a_vm_alloc_share() into gk20a_init_vm().
Remove redundant page size data, and move the page size fields to be
VM specific.
Bug 1558739
Bug 1560370
Change-Id: I4557d9e04d65ccb48fe1f2b116dd1bfa74cae98e
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
gk20a_vm_map_buffer() used to ignore silently map requests for
non-dmabuf fd:s. Fix this by returning the error code from
dma_buf_get().
Bug 1566862
Change-Id: If01b03f43b67b17d9fb997d914db871520f50c6e
Signed-off-by: Sami Kiminki <skiminki@nvidia.com>
When validating buffers to be mapped, check that the buffer end does not
overflow over the virtual address node space.
Bug 1562361
Change-Id: I3c78ec7380584ae55f1e6bf576f524abee846ddd
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
