Account NvMap allocated memory into both RSS and CG tracking to make
efficient OOM kill decisions during memory pressure.
NvMap allocates memory via kernel APIs like alloc_pages, the kernel
memory is not accounted on behalf of process who requests the
allocation. Hence in case OOM, the OOM killer never kills the process
who has allocated memory via NvMap even though this process might be
holding most of the memory.
Solve this issue using following approach:
- Use __GFP_ACCOUNT and __GFP_NORETRY flag
- __GFP_NORETRY will not let the current allocation flow to go into OOM
path, so that it will never trigger OOM.
- __GFP_ACCOUNT causes the allocation to be accounted to kmemcg. So any
allocation done by NvMap will be definitely accounted to kmemcg and
cgroups can be used to define memory limits.
- Add RSS counting for the process which allocates by NvMap, so that OOM
score for that process will get updated and OOM killer can pick this
process based upon the OOM score.
- Every process that has a reference to NvMap Handle would have the
memory size accounted into its RSS. On releasing the reference to
handle, the RSS would be reduced.
Bug 5222690
Change-Id: I3fa9b76ec9fc8d7f805111cb96e11e2ab1db42ce
Signed-off-by: Ketan Patil <ketanp@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/3427871
(cherry picked from commit 2ca91098aa)
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/3453101
Tested-by: Amulya Yarlagadda <ayarlagadda@nvidia.com>
Reviewed-by: Amulya Yarlagadda <ayarlagadda@nvidia.com>
GVS: buildbot_gerritrpt <buildbot_gerritrpt@nvidia.com>
Add more checks in nvmap code so as to avoid any possible races.
- Update is_nvmap_id_ro and is_nvmap_dmabuf_fd_ro functions so that they
return error value during error conditions and also update their callers
to handle those error values.
- Move all trace statements from end of the function to before handle
refcount or dup count is decremented, this make sure we are not
dereferencing any freed handle/reference/dambuf.
- Increment ref's dup count wherever we feel data race is possible, and
decrement it accordingly towards end of function.
Bug 4253911
Change-Id: I50fc7cc98ebbf3c50025bc2f9ca32882138fb272
Signed-off-by: Ketan Patil <ketanp@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/2972602
Reviewed-by: Sachin Nikam <snikam@nvidia.com>
GVS: Gerrit_Virtual_Submit <buildbot_gerritrpt@nvidia.com>
Compression carveout is not correct carveout name as it will be used
by gpu even for non-compression usecases. Hence rename it to gpu
carveout.
Bug 3956637
Change-Id: I802b91d58d9ca120e34655c21f56c0da8c8cf677
Signed-off-by: Ketan Patil <ketanp@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/2955536
Reviewed-by: Pritesh Raithatha <praithatha@nvidia.com>
In NUMA system, user can request nvmap to allocate from any particular
node using NvRmMem APIs. Add code to request allocation from such
requested node. While allocating pages from page pool, check if those
pages belong to requested NUMA node. In some cases, we may not need to
use NUMA support e.g. in color pages allocation code, we don't need to
use NUMA, as page coloring is needed only for t19x chips.
JIRA TMM-5287
Bug 4043165
Change-Id: If8043278122068773a802d83723d287d32914e14
Signed-off-by: Ketan Patil <ketanp@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/2889707
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svc-mobile-cert <svc-mobile-cert@nvidia.com>
Reviewed-by: Sachin Nikam <snikam@nvidia.com>
GVS: Gerrit_Virtual_Submit <buildbot_gerritrpt@nvidia.com>
CBC carveout is not the correct carveout name, rather compression
carveout is the correct name. Compresssion carveout is used to store the
gpu compressible buffers. While the comptags and other metadata related
to these buffers will be stored in CBC carveout which is a GSC carveout
not created/maintained by nvmap. Hence update carveout naming.
Bug 3956637
Change-Id: I50e01a6a8d8bda66960bdee378a12fde176b682a
Signed-off-by: Ketan Patil <ketanp@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nv-oot/+/2888397
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svc-mobile-cert <svc-mobile-cert@nvidia.com>
Reviewed-by: svcacv <svcacv@nvidia.com>
Reviewed-by: Krishna Reddy <vdumpa@nvidia.com>
GVS: Gerrit_Virtual_Submit <buildbot_gerritrpt@nvidia.com>
This patch fixes the following issues:
- When the handle associated with the sub-buffer is freed, the call to
the nvmap_page_pool_fill_lots is made, which would free the pages when
the refcount on the pages is > 1, even though the main handle is not
freed, hence add a check for sub-handle.
- In cpu unmap code, list_for_each_entry is used for iterating over the
vma list and in the same loop list_del is aldo being called, this can
lead to undefined behavior, instead use list_for_each_entry_safe which
is safe against removal of list entry.
- Mutex of sub-handle is not initialized, fix it by initializing it.
- Set error value when handle creation failed.
Bug 3494980
Change-Id: I0659d7f70b44814e87e3081702352e891d9191f7
Signed-off-by: Ketan Patil <ketanp@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvidia/+/2824668
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svc_kernel_abi <svc_kernel_abi@nvidia.com>
Reviewed-by: Puneet Saxena <puneets@nvidia.com>
Reviewed-by: Sachin Nikam <snikam@nvidia.com>
GVS: Gerrit_Virtual_Submit <buildbot_gerritrpt@nvidia.com>
On TOT, in case of carveouts, nvmap performs cache flush operation
during carveout creation, buffer allocation and buffer release. Due to
cache flush for entire carveout at creation time, nvmap takes ~430 ms
for probe. This is affecting boot KPIs.
Fix this by performing cache flush only at buffer allocation time,
instead of carveout creation and buffer release. This is reducing nvmap
probe time to ~0.69 ms.
Bug 3821631
Change-Id: I54da7dd179f8d30b8b038daf3eceafb355b2e789
Signed-off-by: Ketan Patil <ketanp@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvidia/+/2802353
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Reviewed-by: Sachin Nikam <snikam@nvidia.com>
Reviewed-by: Ashish Mhetre <amhetre@nvidia.com>
GVS: Gerrit_Virtual_Submit <buildbot_gerritrpt@nvidia.com>
