In the buddy allocator the BALLOC_PTE_SIZE_* macros are inconsistent
with the gmmu_page_size_* enum. This patch makes the buddy allocator
use the gmmu_page_size_* fields and now has only BALLOC_PTE_SIZE_ANY
for when the allocator does not care about PTE size.
Change-Id: Idbe727b8208e1ace2b947d67f698c471782d5587
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1185136
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Yu-Huan Hsu <yhsu@nvidia.com>
Correct the formula used to determine the range for BE registers
Bug 1778245
Change-Id: I5443b3e68d920cecd031a9b154ed90f26e5251b2
Signed-off-by: Peter Daifuku <pdaifuku@nvidia.com>
Reviewed-on: http://git-master/r/1170602
(cherry picked from commit 813a08f1aa758d718987b4e6f2cf2ac8d15a1611)
Reviewed-on: http://git-master/r/1177828
(cherry picked from commit de8239a5c6241419b98276a5f549ed8cfd7f4cf9)
Reviewed-on: http://git-master/r/1181500
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Richard Zhao <rizhao@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
GVS: Gerrit_Virtual_Submit
When mapping a userspace buffer, determine if it was vidmem allocated
from the aperture of the current gpu, and pass that information into
page tables.
Mapping a vidmem buffer to a gpu it wasn't allocated from is disallowed.
This includes mapping vidmem to igpus and to possibly other dgpus on the
system.
Jira DNVGPU-19
Change-Id: Ia9d2d0133e77659ab96b36ed61eeb4cd5a2b7dff
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1169309
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Add NVGPU_GPU_IOCTL_ALLOC_VIDMEM to the ctrl fd for letting userspace
allocate on-board GPU memory (aka vidmem). The allocations are returned
as dmabuf fds.
Also, report the amount of local video memory in the gpu
characteristics.
Jira DNVGPU-19
Jira DNVGPU-38
Change-Id: I28e361d31bb630b96d06bb1c86d022d91c7592bc
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1181152
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Use the nvgpu-internal buddy allocator for video memory allocations,
instead of nvmap. This allows better integration for copyengine, BAR1
mapping to userspace, etc.
Jira DNVGPU-38
Change-Id: I9fd67b76cd39721e4cd8e525ad0ed76f497e8b99
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1181151
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Added interface to allow kernel to create privileged CE channels for
page migration and clearing support between sysmem and videmem.
JIRA DNVGPU-53
Change-Id: I3e18d18403809c9e64fa45d40b6c4e3844992506
Signed-off-by: Lakshmanan M <lm@nvidia.com>
Reviewed-on: http://git-master/r/1173085
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Unset get_phys_addr_bits as PCIe devices do not need
to care if SMMU is enabled or not.
Jira VFND-1965
Change-Id: I358834863fb365092904da219be80d764f6af8ec
Signed-off-by: Thomas Fleury <tfleury@nvidia.com>
Reviewed-on: http://git-master/r/1183089
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Richard Zhao <rizhao@nvidia.com>
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
In addition to nonwpr_base address, allocate also the wpr_base that is
configured as wpr, in order to not overlap user allocations on that
area.
This patch adds just the wpr region's mem_desc field available. For
devices that use the wpr separately, the field is used for that alloc.
Jira DNVGPU-18
Change-Id: I4a7523f2af2379ee38bdd34cb8ca2d81360dff7b
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1182873
Reviewed-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Implement a bitmap allocator for GPU use. This allocator is useful for
managing memory (or resource) regions where the buddy allocator is not
ideal. Some instances are small regions or where the resource management
must not make calls to the kernel's memory allocation routines (anything
that ultimately calls alloc_page()).
The code path where this avoidance of alloc_page() is most required is
the gpfifo submit path. In order to keep this routine fast and have
predicable time constraints no alloc_page() calls is necessary. The
buddy allocator does not work for this since every time a buddy is
allocated there is the possibility that a pair (or more) buddy structs
have to be made. These allocs could perhaps require a call into
alloc_page() if there is not enouch space in the kmem_cache slab for
the buddy structs.
Change-Id: Ia46fce62d4bdafcebbc153b21b515cb51641d241
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1176446
Reviewed-by: Yu-Huan Hsu <yhsu@nvidia.com>
Allow individual allocacators to be debugged without enabling debugging
on all allocators. The ALLOCATOR_DEBUG define will still work as
expected and enable debugging for all allocators that see this define.
Change-Id: I0d59fa29affeaac15381e65d4128e7bef2f15bd5
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1178689
Reviewed-by: Yu-Huan Hsu <yhsu@nvidia.com>
Move the buddy allocator implementation to a new file to make the
code more organized. Also, as part of this, commonize some macros
and functions which will be used by future allocator implementations.
Bug 1781897
Change-Id: I1611534d5d872bf3b4677f7a1cc024a94b1c437e
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1172116
Reviewed-by: Yu-Huan Hsu <yhsu@nvidia.com>
Support multiple types of allocation backends. Currently there is
only one allocator implementation available: a buddy allocator.
Buddy allocators have certain limitations though. For one the
allocator requires metadata to be allocated from the kernel's
system memory. This causes a given buddy allocation to potentially
sleep on a kmalloc() call.
This patch has been created so that a new backend can be created
which will avoid any dynamic system memory management routines
from being called.
Bug 1781897
Change-Id: I98d6c8402c049942f13fee69c6901a166f177f65
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1172115
GVS: Gerrit_Virtual_Submit
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-by: Yu-Huan Hsu <yhsu@nvidia.com>
When processing FECS traces, a hash table is used
to retrieve the 'pid' of the process that created
the channel/TSG. Report process identifer (aka
tgid in kernel) instead of thread identifier (aka
pid) for FECS traces.
Bug 1736423
Change-Id: I54cb9d298b9fe3e1cccdd7145604cd01c5758c9d
Signed-off-by: Thomas Fleury <tfleury@nvidia.com>
Reviewed-on: http://git-master/r/1166501
(cherry picked from commit f7fd1f6d7ad0753b787ec20604a08a1f4882fe6f)
Reviewed-on: http://git-master/r/1168728
(cherry picked from commit 97a62e5b89352fce576f1bca71b38bf2242ff047)
Reviewed-on: http://git-master/r/1177823
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Richard Zhao <rizhao@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Added a dedicated device node to allow an
app manager to control TSG scheduling parameters:
- Get list of TSGs
- Get list of recent TSGs
- Get list of TSGs per pid
- Get TSG current scheduling parameters
- Set TSG timeslice
- Set TSG runlist interleave
Jira VFND-1586
Change-Id: I014c9d1534bce0eaea6c25ad114cf0cff317af79
Signed-off-by: Thomas Fleury <tfleury@nvidia.com>
Reviewed-on: http://git-master/r/1160384
(cherry picked from commit 75ca739517cc7f7f76714b5f6a1a57c39b8cb38e)
Reviewed-on: http://git-master/r/1167021
Reviewed-by: Richard Zhao <rizhao@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
This patch calculates:
-Total time spent by GPU with rails gated.
-Total time spent by GPU with rails ungated.
-Total Railgating Cycles.
and dumps this information in debugfs file.
This feature requires CONFIG_DEBUG_FS set to true.
Bug 200195100
Change-Id: I1379f11237ce4900076947e18524caaa3304c7cb
Signed-off-by: Deepak Goyal <dgoyal@nvidia.com>
Reviewed-on: http://git-master/r/1178308
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Replace kfree with nvgpu_free in error handling path in
gk20a_alloc_channel_gpfifo where the gpfifo pipe buffer is being
allocated, because it's allocated with nvgpu_alloc.
Jira DNVGPU-21
Change-Id: I73100394b67da2ab064e4e9df6b430d818abce56
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1182401
GVS: Gerrit_Virtual_Submit
Reviewed-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Add "soc/tegra/fuse.h" to include declaration of tegra_get_chip_id()
for kernel version 4.4 and higher as upstream fuse header is not
available in older kernel versions.
Change-Id: Ib83fc6965bc46bb729eab1cc583b9c963f501738
Signed-off-by: Shreshtha SAHU <ssahu@nvidia.com>
Reviewed-on: http://git-master/r/1180686
GVS: Gerrit_Virtual_Submit
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Add static qualifier to nvgpu_pci_pm_init, as it is
only used within the current file.
Bug 200067946
Bug 200088648
Change-Id: Ifb7d3ec174a9f8eea0ac53421c953761886f48c6
Signed-off-by: Sachit Kadle <skadle@nvidia.com>
Reviewed-on: http://git-master/r/1181867
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Sachin Nikam <snikam@nvidia.com>
Tested-by: Sachin Nikam <snikam@nvidia.com>
If page tables are allocated from vidmem, cpu cache flushing doesn't
make sense, so skip it. Unify also map/unmap actions if the pages are
not mapped.
Jira DNVGPU-20
Change-Id: I36b22749aab99a7bae26c869075f8073eab0f860
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1178830
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Enable runtime power management ops for PCIe devices, and
move gk20a_pm_finalize_poweron call into the resume routine.
This change only implements suspend as a stub, as suspend/resume
has not yet been verified for dGPU
Bug 1785512
Bug 200187507
Change-Id: I076bafc03c6b4ba10dce874804ed3fd0b9c7b0d8
Signed-off-by: Sachit Kadle <skadle@nvidia.com>
Reviewed-on: http://git-master/r/1179860
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-by: Thomas Fleury <tfleury@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Richard Zhao <rizhao@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Set has_physical_mode if running on native linux for better performance.
Set it false if running on native gpu but on linux-hv, as the driver
can not get real physical address.
JIRA VFND-1965
Change-Id: I6e0322e64ad14d35d179a33e979157b53d77005a
Signed-off-by: Richard Zhao <rizhao@nvidia.com>
Reviewed-on: http://git-master/r/1175739
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Vladislav Buzov <vbuzov@nvidia.com>
The allocator doesn't give us empty pages, so make sure that they're
full of zeros, just like the sysmem alloc path does.
Jira DNVGPU-16
Change-Id: I0ff8a0718829b13973535ba1111a8a11b91be04d
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1178829
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Add semamphore specific information to the sync_pt_value str when
the underlying primitive for the sync_pt is a semamphore. This is
useful for debugging purposes.
Bug 1732449
JIRA DNVGPU-12
Change-Id: I0dd7d921e39e3245ed1778aad77e20297b55df61
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1162689
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
GVS: Gerrit_Virtual_Submit
For devices that have vidmem available, use the vidmem allocator in
gk20a_gmmu_alloc{,attr,_map,_map_attr}. For others, use sysmem.
Because all of the buffers haven't been tested to work in vidmem yet,
rename calls to gk20a_gmmu_alloc{,attr,_map,_map_attr} to have _sys at
the end to declare explicitly that vidmem is used. Enabling vidmem for
each now is a matter of removing "_sys" from the function call.
Jira DNVGPU-18
Change-Id: Ibe42f67eff2c2b68c36582e978ace419dc815dc5
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1176805
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
We currenlty initialize both runtime PM and pm_domains frameworks
and use pm_domain to control runtime power management of NvGPU
But since GPU has a separate rail, using pm_domain is not
strictly required
Hence remove pm_domain support and use runtime PM only for all
the power management
This also simplifies the code a lot
Initialization in gk20a_pm_init()
- if railgate_delay is set, set autosuspend delay of runtime PM
- try enabling runtime PM
- if runtime PM is now enabled, keep GPU railgated
- if runtime PM is not enabled, keep GPU unrailgated
- if can_railgate = false, disable runtime PM and keep
GPU unrailgated
Set gk20a_pm_ops with below callbacks for runtime PM
static const struct dev_pm_ops gk20a_pm_ops = {
.runtime_resume = gk20a_pm_runtime_resume,
.runtime_suspend = gk20a_pm_runtime_suspend,
.resume = gk20a_pm_resume,
.suspend = gk20a_pm_suspend,
}
Move gk20a_busy() to use runtime checks of pm_runtime_enabled()
instead of using compile time checks on CONFIG_PM
Clean up some pm_domain related code
Remove use of gk20a_pm_enable/disable_clk() since this
should be already done in platform specific unrailgate()/
railgate() APIs
Fix "railgate_delay" and "railgate_enable" sysfs to use
runtime PM calls
For VGPU, disable runtime PM during vgpu_pm_init()
With this, we will initialize vgpu with vgpu_pm_finalize_poweron()
upon first call to gk20a_busy()
Jira DNVGPU-57
Change-Id: I6013e33ae9bd28f35c25271af1239942a4fa0919
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1163216
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Remove code that was compiled out or hard coded not to be ever
invoked.
Coverity ID 24463
Change-Id: Ia4a68bbe43eaebd9f3de1df1318095c014b9e9d0
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1172046
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
When copying chip name to GPU characteristics limit the size of copy
to the size of target name field.
Coverity ID 33613
Change-Id: Ia538d47b9d5e1dd122d57ccd8bfbb3902612874c
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1172007
This function is only used in mm_gk20a.c and as a result should be
static (fixes a sparse issue).
Bug 200088648
Change-Id: I6787b4ebc5925a503d8ef2fed90c3d7cd5027589
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: http://git-master/r/1176309
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Richard Zhao <rizhao@nvidia.com>
Propagate the buffer aperture flag in gk20a_locked_gmmu_map up so that
buffers represented as a mem_desc and present in vidmem can be mapped to
gpu.
JIRA DNVGPU-18
JIRA DNVGPU-76
Change-Id: I46cf87e27229123016727339b9349d5e2c835b3e
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1169308
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Modify page table updates to take an aperture flag (up until
gk20a_locked_gmmu_map()), don't hard-assume sysmem and propagate it to
hardware.
Jira DNVGPU-76
Change-Id: Ifcb22900c96db993068edd110e09368f72b06f69
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1169307
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
add gk20a_aperture_mask() for memory target selection now that buffers
can actually be allocated from vidmem, and use it in all cases that have
a mem_desc available.
Jira DNVGPU-76
Change-Id: I4353cdc6e1e79488f0875581cfaf2a5cfb8c976a
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1169306
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
kmalloc() returns NULL instead of error code on failure. Do not check
if the return value is an error code.
Change-Id: I31a46080ab51773a22bebe4cf03a5b0c94467204
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1172052
Cast cacheline_start to u64 to use 64-bit maths for cacheline offset.
Coverity ID 24250
Change-Id: Ic10c92ebb737bd39486a83e4de53cc1191193667
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: http://git-master/r/1172053
It is possible that when we abort the channel, we have
job clean up worker running, which could race with abort
and sometimes result in below panic
[ 245.483566] Unable to handle kernel paging request at virtual address
800000000
...
[ 245.548991] PC is at gk20a_channel_abort_clean_up+0xb8/0x140
[ 245.554683] LR is at gk20a_channel_abort_clean_up+0xac/0x140
...
[ 247.301860] [<ffffffc000479390>]
gk20a_channel_abort_clean_up+0xb8/0x140
[ 247.312853] [<ffffffc0004794d4>] gk20a_channel_abort+0xbc/0xc8
[ 247.322970] [<ffffffc0004794f8>] gk20a_disable_channel+0x18/0x30
[ 247.333267] [<ffffffc000479628>] gk20a_free_channel+0x118/0x584
[ 247.343473] [<ffffffc000479aa0>] gk20a_channel_close+0xc/0x14
[ 247.353479] [<ffffffc000479b80>] gk20a_channel_release+0xd8/0x104
Fix this by cancelling the job clean up worker before aborting
the channel
Bug 1777281
Change-Id: Ic24c7c03b27cfb5cd164a52efdb1e2813a41a10a
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: http://git-master/r/1174416
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
Currently, host1x power refcount may decrement
to 0, while GPU is still powered on and we're still
servicing IRQs. So to prevent this situation,,
take a ref while GPU is being powered on, and
decrement it during power off. Since we are always
holding one reference while GPU is powered on,
we can remove this handling from gk20a_busy/idle().
Bug 200187507
Change-Id: Idabe88754f009f1e8de8dc821d53be3e013dc657
Signed-off-by: Sachit Kadle <skadle@nvidia.com>
Reviewed-on: http://git-master/r/1172320
(cherry picked from commit 3e27e6a5820f5c1ad05596553d75e8979b71f1bd)
Reviewed-on: http://git-master/r/1172607
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
- $TOP/kernel/nvgpu/drivers/gpu/nvgpu/gm206/acr_gm206.c:207:5:
warning: symbol 'gm206_bootstrap_hs_flcn' was not declared.
Should it be static?
Bug 200067946
Change-Id: Idc8c464b03eb9a382f54095cd3e04786be19308c
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: http://git-master/r/1172969
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
