The last major item preventing the core MM code in the nvgpu
driver from being platform agnostic is the usage of Linux
scattergather tables and scattergather lists. These data
structures are used throughout the mapping code to handle
discontiguous DMA allocations and also overloaded to represent
VIDMEM allocs.
The notion of a scatter gather table is crucial to a HW device
that can handle discontiguous DMA. The GPU has a MMU which
allows the GPU to do page gathering and present a virtually
contiguous buffer to the GPU HW. As a result it makes sense
for the GPU driver to use some sort of scatter gather concept
so maximize memory usage efficiency.
To that end this patch keeps the notion of a scatter gather
list but implements it in the nvgpu common code. It is based
heavily on the Linux SGL concept. It is a singly linked list
of blocks - each representing a chunk of memory. To map or
use a DMA allocation SW must iterate over each block in the
SGL.
This patch implements the most basic level of support for this
data structure. There are certainly easy optimizations that
could be done to speed up the current implementation. However,
this patches' goal is to simply divest the core MM code from
any last Linux'isms. Speed and efficiency come next.
Change-Id: Icf44641db22d87fa1d003debbd9f71b605258e42
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1530867
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
- Added function to read WPR info from FB
MMU registers
- Added HAL to point wpr info read function
- Replaced wpr info read from MC with HAL
- Removed debugfs header include from acr files.
JIRA NVGPU-128
Change-Id: I5ebec46bfe03b9200f2aa569f2e5a780a715616d
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1564683
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Initialize following counters in context header
for all legacy chips:
ctxsw_prog_main_image_num_save_ops
ctxsw_prog_main_image_num_restore_ops
This was already present in the code but move to a function
gk20a_gr_init_ctxsw_hdr_data, so that it can be re-used across
chips.
Additionally initialize following preemption related counters
for gp10b onwards in context header:
ctxsw_prog_main_image_num_wfi_save_ops
ctxsw_prog_main_image_num_cta_save_ops
ctxsw_prog_main_image_num_gfxp_save_ops
ctxsw_prog_main_image_num_cilp_save_ops
Bug 1958308
Change-Id: I0e45ec718a8f9ddb951b52c92137051b4f6a8c60
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1562654
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
GVS: Gerrit_Virtual_Submit
We right now remove a channel from TSG list and disable all the channels in
TSG while removing a channel from TSG
With this sequence if any one channel in TSG is closed, rest of the channels
are set as timed out and cannot be used anymore
We need to fix this sequence as below to allow removing a channel from active
TSG so that rest of the channels can still be used
- disable all channels of TSG
- preempt TSG
- check if CTX_RELOAD is set if support is available
if CTX_RELOAD is set on channel, it should be moved to some other channel
- check if FAULTED is set if support is available
- if NEXT is set on channel then it means channel is still active
print out an error in this case for the time being until properly handled
- remove the channel from runlist
- remove channel from TSG list
- re-enable rest of the channels in TSG
- clean up the channel (same as regular channels)
Add below fifo operations to support checking channel status
g->ops.fifo.tsg_verify_status_ctx_reload
g->ops.fifo.tsg_verify_status_faulted
Define ops.fifo.tsg_verify_status_ctx_reload operation for gm20b/gp10b/gp106
as gm20b_fifo_tsg_verify_status_ctx_reload()
This API will check if channel to be released has CTX_RELOAD set, if yes
CTX_RELOAD needs to be moved to some other channel in TSG
Remove static from channel_gk20a_update_runlist() and export it
Bug 200327095
Change-Id: I0dd4be7c7e0b9b759389ec12c5a148a4b919d3e2
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1560637
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Add platform specific operations to enable/disable a TSG and use them instead
of directly calling enable/disable APIs
For gm20b/gp106/gp10b we continue to use gk20a_enable_tsg() and
gk20a_disable_tsg() as platform specific operations
Bug 1739362
Change-Id: I2dd0f38c8303757e8c7a47d8da0e30a790e514f0
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1560635
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
NVGPU_SUBMIT_GPFIFO_FLAGS_RESCHEDULE_RUNLIST causes host to expire
current timeslice and reschedule from front of runlist.
This can be used with NVGPU_RUNLIST_INTERLEAVE_LEVEL_HIGH to make a
channel start sooner after submit rather than waiting for natural
timeslice expiration or block/finish of currently running channel.
Bug 1968813
Change-Id: I632e87c5f583a09ec8bf521dc73f595150abebb0
Signed-off-by: David Li <davli@nvidia.com>
Reviewed-on: http://git-master/r/#/c/1537198
Reviewed-on: https://git-master.nvidia.com/r/1537198
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Prints addresses of device-specific HAL functions to debugfs file
hal/gops.
The list of functions is produced by dumping the contents of the
gpu_ops substruct of the gk20a struct. This interface makes the
assumption that there are only function pointers in gpu_ops.
Companion Python script nvgpu_debug_hal.py analyzes gk20a.h to
determine operation counts and prettyify debugfs interface's
output.
Jira NVGPU-107
Change-Id: I0910e86638d144979e8630bbc5b330bccfd3ad94
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1542990
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Move non-function pointer members out of the pmu and pmu_ver
substructs of gpu_ops. Ideally gpu_ops will have only function
ponters, better matching its intended purpose and improving
readability.
- g.ops.pmu_ver.cmd_id_zbc_table_update has been changed to
g.pmu_ver_cmd_id_zbc_table_update
- g.ops.pmu.lspmuwprinitdone has been changed to
g.pmu_lsf_pmu_wpr_init_done
- g.ops.pmu.lsfloadedfalconid has been changed to
g.pmu_lsf_loaded_falcon_id
Boolean flags have been implemented using the enabled.h API
- g.ops.pmu_ver.is_pmu_zbc_save_supported moved to
common flag NVGPU_PMU_ZBC_SAVE
- g.ops.pmu.fecsbootstrapdone moved to
common flag NVGPU_PMU_FECS_BOOTSTRAP_DONE
Jira NVGPU-74
Change-Id: I08fb20f8f382277f2c579f06d561914c000ea6e0
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1530981
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reorganize HAL initialization to remove inheritance and construct
the gpu_ops struct at compile time. This patch only covers the
mm sub-module of the gpu_ops struct.
Perform HAL function assignments in hal_gxxxx.c through the
population of a chip-specific copy of gpu_ops.
Jira NVGPU-74
Change-Id: Ieb87a62f047510e51c52e6563d8e3fd5a65b5f28
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1537753
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reorganize HAL initialization to remove inheritance and construct
the gpu_ops struct at compile time. This patch only covers the
mm sub-module of the gpu_ops struct.
Perform HAL function assignments in hal_gxxxx.c through the
population of a chip-specific copy of gpu_ops.
Jira NVGPU-74
Change-Id: I289284e6e528fc7951c959c8765ccf9349eec33b
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1533351
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Remove the mm.get_iova_addr() HAL and replace it with a new HAL
called mm.gpu_phys_addr(). This new HAL provides the real phys
address that should be passed to the GPU from a physical address
obtained from a scatter list. It also provides a mechanism by
which the HAL code can add extra bits to a GPU physical address
based on the attributes passed in. This is necessary during GMMU
page table programming.
Also remove the flags argument from the various address functions.
This flag was used for adding an IO coherence bit to the GPU
physical address which is not supported.
JIRA NVGPU-30
Change-Id: I69af5b1c6bd905c4077c26c098fac101c6b41a33
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1530864
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Pass struct gk20a pointer instead of struct device to
gk20a_wait_for_idle(). The code is not Linux specific and does not
need pointer to struct device.
Change-Id: I2cafd6c7db019c9de76b6e68a1ae73f0b4cea37d
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1533173
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Refactor the sync_debugfs LTC HAL op so that the logic to enable
or disable LTC goes to common code nvgpu_ltc_sync_enabled() and
the LTC HAL set_enabled only performs the hardware register access.
Create a new common function nvgpu_init_ltc_support() to initialize
the LTC software variable, and move hardware initialization of LTC to
be called from it.
JIRA NVGPU-62
Change-Id: Ib1cf4f5b83ca3dac08407464ed56a732e0a33923
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1528262
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Move fields in struct gk20a related to interrupt handling into
Linux specific nvgpu_os_linux. At the same time move the counter
logic from function in HAL into Linux specific code, and two Linux
specific power management functions from generic gk20a.c to Linux
specific module.c.
JIRA NVGPU-123
Change-Id: I0a08fd2e81297c8dff7a85c263ded928496c4de0
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1528177
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Sourab Gupta <sourabg@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reorganize HAL initialization to remove inheritance and construct
the gpu_ops struct at compile time. This patch only covers the
gr_ctx sub-module of the gpu_ops struct.
Perform HAL function assignments in hal_gxxxx.c through the
population of a chip-specific copy of gpu_ops.
Jira NVGPU-74
Change-Id: I783d8e8919d8694ad2aa0d285e4c5a2b62580f48
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1527417
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Replace privsecurity boolean flag in gpu_ops with entry in
common flag system.
The new common flag is NVGPU_SEC_PRIVSECURITY
Jira NVGPU-74
Change-Id: I4b258f5ffbe30a6344ffba0ece51c6f5d47ebec1
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1525713
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Alex Waterman <alexw@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Replace securegpccs boolean flag in gpu_ops with entry in
common flag system.
The new common flag is NVGPU_SEC_SECUREGPCCS
Jira NVGPU-74
Change-Id: I46430f95063f617531cf0e5aba472051b41f4a9d
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1514060
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Alex Waterman <alexw@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Replace pmupstate boolean flag in gpu_ops with entry in
common flag system.
The new common flag is NVGPU_PMU_PSTATE
Jira NVGPU-74
Change-Id: I4e1ba922a48145b1cf3488b6f14fde78107adb5b
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1514059
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Alex Waterman <alexw@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
- add is_egpc_addr, is_etpc_addr and get_egpc_etpc_num gr ops
- add gr ops for decode and create egpc/etpc priv addr
- add etpc as part of ctxsw_regs
JIRA GPUT19X-49
Bug 200311674
Bug 1960226
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Change-Id: I9a8be1804a9354238de2441093b3b136321b7e53
Reviewed-on: https://git-master.nvidia.com/r/1522442
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
The new SET_BES_CROP_DEBUG3 sw method is used to flip two fields
in the NV_PGRAPH_PRI_BES_CROP_DEBUG3 register. The sw method is
used by the user space driver to disable enough ROP optimizations
to maintain ZBC state of target tiles.
Bug 1942454
Change-Id: Id4e4d9d06c6c66080d06b6d4694546fe5cba8436
Signed-off-by: Lauri Peltonen <lpeltonen@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1516202
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Move non-function pointer members out of the mc substruct of
gpu_ops. Ideally gpu_ops will have only function ponters, better
matching its intended purpose and improving readability.
gops.mc.intr_mask_restore is now mc_intr_mask_restore
Jira NVGPU-74
Change-Id: Iddf5e761012086e993c375d59e1bbdfb1e64db0c
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1509602
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
A negative value in the timeout duration does not have any special uses,
so change the duration type to u32 (from just int). Delete some
unnecessary typecasts to int.
Also change MAX_SCHEDULE_TIMEOUT to ULONG_MAX in default gr idle timeout
because the value is in milliseconds instead of scheduling units and to
drop unnecessary Linux dependency.
Change-Id: I5cf6febd4f1cb00c46fe159603436a9ac3b003ac
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master/r/1512565
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
On some GPUs certain physical address bits have special meaning. This
patch adds support for setting those bits based on the GMMU attributes
struct.
Jira GPUT19X-10
Bug 200279508
Change-Id: I32b8a028be7fd62af06a60c393a8c9251de0ef3c
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master/r/1512600
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Add lock_down_sm and wait_for_sm_lock_down gr ops
Required to support multiple SM and t19x SM register
address changes
JIRA GPUT19X-75
Change-Id: I529babde51d9b2143fe3740a4f67c582b7eb404b
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master/r/1514042
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
This is required to take care of t19x changes to support
multiple SM
JIRA GPUT19X-75
Change-Id: Ifd2cb28ae442462fef1d2c4439baa817f00c2c9e
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master/r/1514041
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
When two or more apps ran simultaneously
First app context sets power_on flag & starts init
Other app context check the power_on flag
and try to use GPU without init completed
Which makes aother apps to assert
Added mutex to synchronize poweron access
Bug 200297265
Change-Id: Ie138f7f43bb0dd3304ed91ae3649a6a4947bee91
Signed-off-by: skadamati <skadamati@nvidia.com>
Reviewed-on: https://git-master/r/1511436
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
This is required to support multiple SM and t19x
sm register address changes
JIRA GPUT19X-75
Change-Id: I844b5cf02a75ba397891a1100d917875e5a3e181
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master/r/1512217
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
This is required to support multiple SM and t19x
sm register address changes
JIRA GPUT19X-75
Change-Id: If8805bcc042c75ea70c1689306feb3c8bf011655
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master/r/1512216
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
This is required to support multiple SM and t19x
sm register address changes
JIRA GPUT19X-75
Change-Id: Icdae3b6ed67a3d3deeb17f29528184b2d7a70af5
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master/r/1512215
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
