On TOT, NvMap does page by page cache flush i.e. it takes virtual
address of each page present in the buffer and then perform cache
flush on it using dcache_by_line_op. This result in very poor
performance for larger buffers. ~70% of the time taken by
NvRmMemHandleAllocAttr is consumed in cache flush.
Address this perf issue using multithreaded cache flush
- Use a threshold value of 32768 pages which is derived from perf
experiments and as per discussion with cuda as per usecases.
- When the cache flush request of >= 32768 pages is made, then vmap
pages to map them in contiguous VA space and create n number of kernel
threads; where n indicate the number of online CPUs.
- Divide the above VA range among the threads and each thread would do
cache flush on the VA range assigned to it.
This logic in resulting into following % improvement for alloc tests.
-----------------------------------
Buffer Size in MB | % improvement |
----------------------------------|
128 | 52 |
256 | 56 |
512 | 57 |
1024 | 58 |
1536 | 57 |
2048 | 58 |
2560 | 57 |
3072 | 58 |
3584 | 58 |
4096 | 58 |
4608 | 58 |
5120 | 58 |
-----------------------------------
Bug 4628529
Change-Id: I803ef5245ff9283fdc3afc497a6b642c97e89c06
Signed-off-by: Ketan Patil <ketanp@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/3187871
Reviewed-by: Krishna Reddy <vdumpa@nvidia.com>
GVS: buildbot_gerritrpt <buildbot_gerritrpt@nvidia.com>
In Linux v6.13, commit b129125e1f96 ("arm64: asm-offsets: remove
DMA_{TO,FROM}_DEVICE") removes the definitions DMA_TO/FROM_DEVICE from
the assembly header file 'asm-offsets.h' which is used by the functions
__dma_map_area() and __dma_unmap_area() in nvmap_cache_maint.S.
From reviewing the NVMAP code, the function __dma_unmap_area() is never
used and so we can simply remove this. Split the __dma_map_area() into
two functions __dma_map_area_to_device() and
__dma_map_area_from_device() to avoid having to pass the direction.
Bug 4991895
Change-Id: I4a0f658401beff5c5e5457de72050b21acf820fa
Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/3261710
GVS: buildbot_gerritrpt <buildbot_gerritrpt@nvidia.com>
Reviewed-by: Ketan Patil <ketanp@nvidia.com>
- Add nvmap_dev.h file which include declaration for functions and data
structs which are exposed by nvmap_dev unit to other units.
- Also, add nvmap_dev_int.h file which include declaration for
functions which are internal to nvmap_dev unit that can be called by
files within nvmap_dev unit.
- Move definition of nvmap_handle_get_from_id, nvmap_install_fd,
find_range_of_handles to nvmap_handle.c as they belong to nvmap_handle
unit.
- Cleanup nvmap_priv.h by moving all relevant items for nvmap_dev unit
to nvmap_dev unit.
- Remove nvmap_mm.c file as nvmap_zap_handle is the only function
present; move it to nvmap_cache.c where it's being called.
- Remove function declarations whose definition are not present.
JIRA TMM-5694
JIRA TMM-5730
Change-Id: Ifd45235076da2ef0c628f3179d828c0ccadf6df2
Signed-off-by: Ketan Patil <ketanp@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/3223994
Reviewed-by: Pritesh Raithatha <praithatha@nvidia.com>
GVS: buildbot_gerritrpt <buildbot_gerritrpt@nvidia.com>
- Files for nvmap_handle unit: nvmap_handle.c, nvmap_sci_ipc.c,
nvmap_id_array.c.
- Define external header for nvmap_handle unit as nvmap_handle.h and
move declarations of all external APIs of nvmap_handle unit to this
header.
- Define internal header for nvmap_handle unit as nvmap_handle_int.h and
move declarations of all internally called APIs to this header.
JIRA TMM-5651
Change-Id: Ie4922c0839070491f9893f23744eb700cabb9828
Signed-off-by: Ashish Mhetre <amhetre@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/3211591
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
- Move macro definitions from nvmap_priv.h to nvmap_alloc unit wherever
required.
- Cleanup unnecessary macros.
- Add function to cleanup the memory allocated for debugfs_info for
iovmm. This was missed in the previous patch where the allocation for
debugfs_info is moved to dynamic memory allocation.
- Move nvmap page pool related data structs from nvmap_priv to
nvmap_alloc unit.
JIRA TMM-5621
Change-Id: I3b668b2d6182da1bf0d2034c66834efc02d3179f
Signed-off-by: Ketan Patil <ketanp@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/3203118
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Fix for Rule: Misra-C 2012 Rule 10.1
- Data type of client is a pointer, hence compare it with NULL.
- Data type of co is a pointer, hence compare it with NULL.
- h is a pointer, hence compare with NULL.
- size is of data type size_t, hence compare it with OU.
- Data type of node is a pointer, hence compare it with NULL.
- Data type of priv, priv->handle is a pointer,
hence compare with NULL.
- Data value of Kzalloc is pointer not a boolean, hence compare new with NULL.
- Data type of vma is a pointer, hence compare it with NULL.
- Data type of elem_size and count is unsigned long, hence compare it
with 0.
- Data type of nr is u32, hence compare it with 0U.
- Data type of ret is int, hence compare it with 0.
- Data type of heap is pointer, hence compare it with NULL.
CID 1675220
CID 1677129
CID 1680522
CID 1680855
CID 1682355
CID 1684748
CID 1685031
CID 1688104
CID 1691439
CID 1691492
CID 1697576
CID 1700206
CID 1703733
CID 1705732
CID 1713149
CID 1713881
CID 1715301
CID 1716395
CID 1718186
CID 1724356
CID 1736224
CID 1737251
CID 1742375
CID 1742507
CID 1743460
CID 1747820
CID 1751065
CID 1753197
CID 1754913
CID 1756020
CID 1758334
CID 1761585
CID 1762790
CID 1763725
JIRA TMM-5594
Change-Id: Iec045c45555b364b5869de856b9bb8a8586dfe02
Signed-off-by: Surbhi Singh <surbhis@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/3201341
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
As part of the nvmap_refactoring, add nvmap_alloc.h file which include
declaration for functions which are exposed by nvmap_alloc unit to other
units. Also, add nvmap_alloc_int.h file which include declaration for
functions which are internal to nvmap_alloc unit that can be called by
files within nvmap_alloc unit.
JIRA TMM-5621
Change-Id: Ie30e5e8a4f87591eb9c49a0a349f837a22726fa5
Signed-off-by: Ketan Patil <ketanp@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/3198546
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Remove macro-protected dead code. NVMAP_LOADABLE_MODULE enables NvMap as
an OOT module, while NVMAP_UPSTREAM_KERNEL is a config for kstable/OOT
kernel. For Kernel 5.10+ both are always defined, So the related macro
protected code can be safely removed.
Bug 4479135
Change-Id: I792f1cb2c54fd21bcf0e73ffc52e46e4efd47862
Signed-off-by: Yash Bhatt <ybhatt@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/3079420
Reviewed-by: Ashish Mhetre <amhetre@nvidia.com>
Reviewed-by: Sachin Nikam <snikam@nvidia.com>
GVS: Gerrit_Virtual_Submit <buildbot_gerritrpt@nvidia.com>
The function ioremap_prot() has been exported by the Linux kernel since
v5.9. However, it is only available if GENERIC_IOREMAP is enabled. Since
Linux v6.0, when the ARM64 __ioremap() function was removed, was
GENERIC_IOREMAP enabled for ARM64 by default. Rather than depend on
kernel version for selecting if ioremap_prot is used, use ioremap_prot
if GENERIC_IOREMAP is enabled.
This is a better solution for working with 3rd party Linux kernels that
may backport upstream changes to the kernel and hence are not compatible
with these existing version checks.
Bug 4119327
Change-Id: I2f5f9a691c66c5fbd83d4aa51062c749d39dd447
Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/2982126
Reviewed-by: Ketan Patil <ketanp@nvidia.com>
GVS: Gerrit_Virtual_Submit <buildbot_gerritrpt@nvidia.com>
__ioremap function called during cache maint takes more time in case of
multiple processes compared to single process. This is impacting camera
tests when multiple cameras are used. For carveouts, nvmap performs page
by page ioremap and then cache maint on it. Optimize this with following
steps:
- During first cache_maint of the buffer, perform ioremap on entire
buffer and save the kernel V.A.
- Next time, when cache_maint is called on the same buffer, we will
reuse the saved kernel V.A.
Bug 3922175
Change-Id: Iecb372365409de1951bcc8e6f3d8e8e867bb2a80
Signed-off-by: Ketan Patil <ketanp@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/2948759
Reviewed-by: Barton Xu (SW-TEGRA) <bartonx@nvidia.com>
Reviewed-by: Krishna Reddy <vdumpa@nvidia.com>
Reviewed-by: Sachin Nikam <snikam@nvidia.com>
GVS: Gerrit_Virtual_Submit <buildbot_gerritrpt@nvidia.com>
