Add api to translate dmabuf's fmode_t to gk20a_mem_rw_flag
for read only/read write mapping selection.
By default dmabuf fd mapping permission should be a maximum
access permission associated to a particual dmabuf fd.
Remove bit flag MAP_ACCESS_NO_WRITE and add 2 bit values for
user access requests NVGPU_VM_MAP_ACCESS_DEFAULT|READ_ONLY|
READ_WRITE.
To unify map access type handling in Linux and QNX move the
parameter NVGPU_VM_MAP_ACCESS_* check to common function
nvgpu_vm_map.
Set MAP_ACCESS_TYPE enabled flag in common characteristics
init function as it is supported for Linux and QNX.
Bug 200717195
Bug 3250920
Change-Id: I1a249f7c52bda099390dd4f371b005e1a7cef62f
Signed-off-by: Lakshmanan M <lm@nvidia.com>
Signed-off-by: Sagar Kamble <skamble@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2507150
Reviewed-by: svc_kernel_abi <svc_kernel_abi@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svc-mobile-misra <svc-mobile-misra@nvidia.com>
Reviewed-by: svc-mobile-cert <svc-mobile-cert@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Add new MAPPING_MODIFY ioctl to the linux nvgpu driver.
This ioctl is used (for example) by the NvRmGpuMappingModify API to
change the kind of an existing mapping.
For compressed mappings the ioctl can be used to do the following:
* switch between two different compressed kinds
* switch between compressed and incompressed kinds
For incompressed mappings the ioctl can be used to do the following:
* switch between two different incompressed kinds
In order to properly update an existing mapping the nvgpu_mapped_buf
structure has been extended to cache the following state when the
mapping is first created:
* the compression tag offset (if applicable)
* the GMMU read/write flags
* the memory aperture
The unused ctag_lines field in the nvgpu_ctag_buffer_info structure
has been replaced with a new ctag_offset field.
Jira NVGPU-6374
Change-Id: I647ab9c2c272e3f9b52f1ccefc5e0de4577c14f1
Signed-off-by: scottl <scottl@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2468100
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svc-mobile-misra <svc-mobile-misra@nvidia.com>
Reviewed-by: svc-mobile-cert <svc-mobile-cert@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
GVS: Gerrit_Virtual_Submit
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Until now, all userspace buffers were mapped in the GMMU as Read & Write
(RW) by default. In order to enable the use cases which require the GPU
to only read the SYSMEM buffers and not inadvertently write to those,
map buffer ioctls need to provide interface to set the mapping access
type from the userspace.
Some of the use cases are:
1. A third party server process exposes shared memory that is
read-only to the client process, which does the GPU processing.
Registering this memory using cudaHostRegister API as read-only
in the client process will restict the access to Read Only type
from the GPU.
2. IO devices exposing streaming read-only data for processing by
the GPU.
3. For marking semantically read-only data as actually read-only
for the purposes of debugging data corruption.
This patch introduces new AS buffer mapping bitmask flag and
corresponding core VM mapping bitmask flag for representing
Read Only (RO) access type. By default, the access is set
as Read Write (RW).
Bug 200621157
Change-Id: I5ec9dec3ce089e577b86c43003d92b61eee4a90b
Signed-off-by: Sagar Kamble <skamble@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2361750
Reviewed-by: automaticguardword <automaticguardword@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
GVS: Gerrit_Virtual_Submit
The function parameter of nvgpu_vm_map function is fixed for MISRA
where implicit assignment of objects to a narrower or different
essential type not allowed.This fixes few enum violations.
JIRA NVGPU-1584
Change-Id: I2353f7501c3326f792f5942b2e247badf03349cf
Signed-off-by: Dinesh T <dt@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1986509
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Update the Linux specific code to match the MM API docs in the
previous patch. The user passed page size is plumbed through
the Linux VM mapping calls but is ultimately ignored once the
core VM code is called. This will be handled in the next
patch.
This also adds some code to make the CDE page size picking
happen semi-intelligently. In many cases the CDE buffers can
be mapped with large pages.
Bug 2011640
Change-Id: I20e78e7d5a841e410864b474179e71da1c2482f4
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1740610
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
VM and CDE code assumes that dma_buf_attachment is stored as a pointer
in the private dma_buf_drvdata, so it is not tracked. In Linux trees
without dma_buf_*_drvdata() support this is not true, so change the
code to explicitly track dma_buf_attachment.
JIRA NVGPU-4
Change-Id: I692f05a19a6469195d5444a7e5ff6e92f77ae272
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1648004
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Final VM mapping refactoring. Move most of the logic in the VM
map path to the common/mm/vm.c code and use the generic APIs
previously implemented to deal with comptags and map caching.
This also updates the mapped_buffer struct to finally be free
of the Linux dma_buf and scatter gather table pointers. This
is replaced with the nvgpu_os_buffer struct.
JIRA NVGPU-30
JIRA NVGPU-71
JIRA NVGPU-224
Change-Id: If5b32886221c3e5af2f3d7ddd4fa51dd487bb981
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1583987
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Add a generic nvgpu_os_buffer type, defined by each OS, to abstract
a "user" buffer. This allows the comptag interface to be used in the
core code.
The end goal of this patch is to allow the OS specific mapping code
to call a generic mapping function that handles most of the mapping
logic. The problem is a lot of the logic involves comptags which are
highly dependent on the operating systems buffer management scheme.
With this, each OS can implement the buffer comptag mechanics
however it wishes without the core MM code caring.
JIRA NVGPU-30
JIRA NVGPU-223
Change-Id: Iaf64bc52e01ef3f262b4f8f9173a84384db7dc3e
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1583986
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Since NVGPU_AS_MAP_BUFFER_FLAGS_DIRECT_KIND_CTRL was made mandatory,
kernel does not need to know the details about the PTE kinds
anymore. Thus, we can remove the kind_gk20a.h header and the code
related to kind table setup, as well as simplify buffer mapping code
a bit.
Bug 1902982
Change-Id: Iaf798023c219a64fb0a84da09431c5ce4bc046eb
Signed-off-by: Sami Kiminki <skiminki@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1560933
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
The function nvgpu_vm_map_linux() used to return GPU VA on successful
map or 0 when things didn't go smoothly. However, this scheme does not
propagate the actual map error back to the userspace.
So, modify the function a bit: return error and return the GPU VA via
pointer on success.
Bug 1705731
Change-Id: I2174b5fbaf64dcb00f9567dab1c583d6ddfa5d78
Signed-off-by: Sami Kiminki <skiminki@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1590961
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Remove the buffer_attrs struct and replace it with a more
streamlined nvgpu_ctag_buffer_info struct. This struct allows
several different fields to all be passed by pointer to the
various kind/compression functions in the VM map process.
This path also moves several comptag/kind related functions
to the core vm.c code since these functions can be reused by
other OSes.
Change-Id: I2a0f0a1c4b554ce4c8f2acdbe3161392e717d3bf
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1583984
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Remove the always true field 'user_mapped' from the mapped_buf
struct. All mapped_bufs are mapped by a user request since they
always originate from a dma_buf (for Linux, that is). As such
there is a fair amount of logic that could be deleted.
Linux specific: the own_mem_ref field was also be deleted. The
logic of only storing a dma_buf ref when the buffer is mapped
for the first time by a user is easy: when the mapped buffer is
found in the map cache release the outstanding dma_buf ref taken
earlier on in the map path. If the map cache does not have the
buffer simply let the higher level map code keep the dma_buf ref.
The dma_buf ref is released when the nvgpu_vm_unmap_system()
call-back is called by the unmap path.
JIRA NVGPU-30
JIRA NVGPU-71
Change-Id: I229d136713812a7332bdadd5ebacd85d983bbbf0
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1583983
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Re-organize the unmap code to be better split between OS specific
requirements and common core requirements. The new code flow works
as follows:
nvgpu_vm_unmap()
Is the primary entrance to the unmap path. It takes a VM and a GPU
virtual address to unmap. There's also an optional batch mapping
struct.
This function is responsible for making sure there is a real buffer
and that if it's being called on a fixed mapping then the mapping
will definitely be freed (since buffers are ref-counted). Then this
function decrements the ref-count and returns.
If the ref-count hits zero then __nvgpu_vm_unmap_ref() is called
which just calls __nvgpu_vm_unmap() with the relevant batch struct
if present. This is where the real work is done. __nvgpu_vm_unmap()
clears the GMMU mapping, removes the mapped buffer from the various
lists and trees it may be in and then calls the
nvgpu_vm_unmap_system() function. This function handles any OS
specific stuff and must be defined by all VM OS implementations.
There's a a short cut used by some other core VM code to free
mappings without going through nvgpu_vm_map(). Mostly they just
directly decrement the mapping ref-count which can then call
__nvgpu_vm_unmap_ref() if the ref-count hits zero.
JIRA NVGPU-30
JIRA NVGPU-71
Change-Id: Ic626d37ab936819841bab45214f027b40ffa4e5a
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1583982
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Delte the Linux headers and make some modifications to get rid of the
minor compilation issues that resulted.
- Add <linux/iommu.h> to os_linux.h
- Delete #if 0 code that "flushed" a buffer in gr_gk20a.c
- Delete FLUSH_CPU_DCACHE() macro
- Move the cache flush definitions to <nvgpu/linux/vm.h>
and include this header in sim_gk20a.c. This file will
not be used by QNX so this should be fine.
- Add <linux/pci_ids.h> to gp106/bios_gp106.c and
gp106/mclk_gp106.c.
- Move function to common/linux/dmabuf.h since it is a
dmabuf related function and uses a struct device pointer
as an argument.
JIRA NVGPU-30
Change-Id: I11f56b98524c7fac3efa91b4686592130e5f8a46
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1585510
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Move vm_priv.h to <nvgpu/linux/vm.h> and rename nvgpu_vm_map()
to nvgpu_vm_map_linux(). Also remove a redundant unmap function
from the unmap path. These changes are the beginning of reworking
the nvgpu Linux mapping and unmapping code.
The rest of this patch is just the necessary changes to use the
new map function naming and the new path to the Linux vm header.
Patch Series Goal
-----------------
There's two major goals for this patch series. Note that these
goals are not achieved in this patch. There will be subsequent
patches.
1. Remove all last vestiges of Linux code from common/mm/vm.c
2. Implement map caching in the common/mm/vm.c code
To accomplish this firstly the VM mapping code needs to have the
struct nvgpu_mapped_buf data struct be completely Linux free. That
means implementing an abstraction for this to hold the Linux stuff
that mapped buffers carry about (SGT, dma_buf). This is why the
vm_priv.h code has been moved: it will need to be included by the
<nvgpu/vm.h> header so that the OS specific struct can be pulled
into struct nvgpu_mapped_buf.
Next renaming the nvgpu_vm_map() to nvgpu_vm_map_linux() is in
preparation for adding a new nvgpu_vm_map() that handles the
map caching with nvgpu_mapped_buf. The mapping code is fairly
straight forward: nvgpu_vm_map does OS generic stuff; each OS
then calls this function from an nvgpu_vm_map_<OS>() or the like
that does any OS specific adjustments/management.
Freeing buffers is much more tricky however. The maps are all
reference counted since userspace does not track buffers and
expects us to handle this instead. Ugh! Since there's ref-counts
the free code will require a callback into the OS specific code
since the OS specific code cannot free a buffer directly. THis
make's the path for freeing a buffer quite convoluted.
JIRA NVGPU-30
JIRA NVGPU-71
Change-Id: I5e0975f60663a0d6cf0a6bd90e099f51e02c2395
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1578896
GVS: Gerrit_Virtual_Submit
Reviewed-by: David Martinez Nieto <dmartineznie@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
