Commit Graph

1921 Commits

Author SHA1 Message Date
David Nieto
7134e9e852 gpu: nvgpu: prevent crash during unbind
This change solves crashes during bind that were introduced in the driver
during the OS unification refactoring due to lack of coverage of the remove()
function.

The fixes during remove are:

(1) Prevent NULL dereference on GPUs with secure boot
(2) Prevent NULL dereferences when fecs_trace is not enabled
(3) Added PRAMIN blocker during driver removal if HW is no longer accesible
(4) Prevent double free of debugfs nodes as they are handled on the
debugfs_remove_recursive() call
(5) quiesce() can now be called without checking is HW accesible flag is set
(6) added function to free irq so no IRQ association is left on the driver after
it is removed
(7) prevent NULL dereference on nvgpu_thread_stop() if the thread is already
stopped

JIRA: EVLR-1739

Change-Id: I787d38f202d5267a6b34815f23e1bc88110e8455
Signed-off-by: David Nieto <dmartineznie@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1563005
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-09-22 15:44:25 -07:00
Sunny He
17c581d755 gpu: nvgpu: SGL passthrough implementation
The basic nvgpu_mem_sgl implementation provides support
for OS specific scatter-gather list implementations by
simply copying them node by node. This is inefficient,
taking extra time and memory.

This patch implements an nvgpu_mem_sgt struct to act as
a header which is inserted at the front of any scatter-
gather list implementation. This labels every struct
with a set of ops which can be used to interact with
the attached scatter gather list.

Since nvgpu common code only has to interact with these
function pointers, any sgl implementation can be used.
Initialization only requires the allocation of a single
struct, removing the need to copy or iterate through the
sgl being converted.

Jira NVGPU-186

Change-Id: I2994f804a4a4cc141b702e987e9081d8560ba2e8
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1541426
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-09-22 12:55:24 -07:00
Alex Waterman
0090ee5aca gpu: nvgpu: nvgpu SGL implementation
The last major item preventing the core MM code in the nvgpu
driver from being platform agnostic is the usage of Linux
scattergather tables and scattergather lists. These data
structures are used throughout the mapping code to handle
discontiguous DMA allocations and also overloaded to represent
VIDMEM allocs.

The notion of a scatter gather table is crucial to a HW device
that can handle discontiguous DMA. The GPU has a MMU which
allows the GPU to do page gathering and present a virtually
contiguous buffer to the GPU HW. As a result it makes sense
for the GPU driver to use some sort of scatter gather concept
so maximize memory usage efficiency.

To that end this patch keeps the notion of a scatter gather
list but implements it in the nvgpu common code. It is based
heavily on the Linux SGL concept. It is a singly linked list
of blocks - each representing a chunk of memory. To map or
use a DMA allocation SW must iterate over each block in the
SGL.

This patch implements the most basic level of support for this
data structure. There are certainly easy optimizations that
could be done to speed up the current implementation. However,
this patches' goal is to simply divest the core MM code from
any last Linux'isms. Speed and efficiency come next.

Change-Id: Icf44641db22d87fa1d003debbd9f71b605258e42
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1530867
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-09-22 12:52:48 -07:00
Mahantesh Kumbar
e32cc0108c gpu: nvgpu: read WPR info from fb
- Added function to read WPR info from FB
MMU registers
- Added HAL to point wpr info read function
- Replaced wpr info read from MC with HAL
- Removed debugfs header include from acr files.

JIRA NVGPU-128

Change-Id: I5ebec46bfe03b9200f2aa569f2e5a780a715616d
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1564683
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
2017-09-22 06:14:02 -07:00
Alex Waterman
b9082f0760 gpu: nvgpu: Add per-GPU total to DMA memory prints
Track the total amount of DMA memory currently outstanding for each
GPU. Print this total in the DMA debugging/logging prints.

Bug 1956137

Change-Id: I929598e5aa388ee84db0badb4eb9f7c6cbe030c7
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1559518
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-09-19 21:52:56 -07:00
seshendra Gadagottu
1132fd2a12 gpu: nvgpu: changes related handling ctx header
ctx header holds only gpu va for each address space.
All other information will be held in main
context. Ctx header will have gpu va for following
fields:
ctxsw_prog_main_image_context_buffer_ptr
ctxsw_prog_main_image_context_buffer_ptr_hi
ctxsw_prog_main_image_zcull_ptr
ctxsw_prog_main_image_zcull_ptr
ctxsw_prog_main_image_pm_ptr
ctxsw_prog_main_image_pm_ptr_hi
ctxsw_prog_main_image_full_preemption_ptr_hi
ctxsw_prog_main_image_full_preemption_ptr
ctxsw_prog_main_image_full_preemption_ptr_xxxx0
ctxsw_prog_main_image_full_preemption_ptr_xxxx0_v
ctxsw_prog_main_image_patch_adr_lo
ctxsw_prog_main_image_patch_adr_hi

Changes done as part of this CL:
- Read ctx_id from from main context header
- Golden context creation:
  Use gold_mem for for golden context creation
  and copy golden context from save gold local
  memory to main context. No need to restore
  golden context to context header.
- Write ctx_patch_count and smpc_ctxsw_mode in
  main context header only.
- Update preemption mode in main context header and
  preemption buffer va in context header.
- Updated image patch buffer va in context header.

Bug 1958308

Change-Id: Ic076aad8b1802f76f941d2d15cb9a8c07308e3e8
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1562680
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
GVS: Gerrit_Virtual_Submit
2017-09-19 17:45:29 -07:00
seshendra Gadagottu
c4370d7def gpu: nvgpu: Initialize ctxsw header counters
Initialize following counters in context header
for all legacy chips:
ctxsw_prog_main_image_num_save_ops
ctxsw_prog_main_image_num_restore_ops

This was already present in the code but move to a function
gk20a_gr_init_ctxsw_hdr_data, so that it can be re-used across
chips.

Additionally initialize following preemption related counters
for gp10b onwards in context header:
ctxsw_prog_main_image_num_wfi_save_ops
ctxsw_prog_main_image_num_cta_save_ops
ctxsw_prog_main_image_num_gfxp_save_ops
ctxsw_prog_main_image_num_cilp_save_ops

Bug 1958308

Change-Id: I0e45ec718a8f9ddb951b52c92137051b4f6a8c60
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1562654
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
GVS: Gerrit_Virtual_Submit
2017-09-19 17:45:28 -07:00
Seema Khowala
e229cb76ba gpu: nvgpu: fix verbose return value
verbose return value is not taking all the channels into account.
Fix this by ORing verbose values for all channels.

Change-Id: Id77c74458067c72792422aa69be1626c3d164e1c
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1549645
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-09-18 18:19:07 -07:00
Sami Kiminki
5d09c908b0 gpu: nvgpu: Direct GMMU PTE kind control
Allow userspace to control directly the PTE kind for the mappings by
supplying NVGPU_AS_MAP_BUFFER_FLAGS_DIRECT_KIND_CTRL for MAP_BUFFER_EX.

In particular, in this mode, the userspace will tell the kernel
whether the kind is compressible, and if so, what is the
incompressible fallback kind. By supplying only the compressible kind,
the userspace can require that the map kind will not be demoted to the
incompressible fallback kind in case of comptag allocation failure.

Add also a GPU characteristics flag
NVGPU_GPU_FLAGS_SUPPORT_MAP_DIRECT_KIND_CTRL to signal whether direct
kind control is supported.

Fix indentation of nvgpu_as_map_buffer_ex_args header comment.

Bug 1705731

Change-Id: I317ab474ae53b78eb8fdd31bd6bca0541fcba9a4
Signed-off-by: Sami Kiminki <skiminki@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1543462
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-09-15 15:45:45 -07:00
Deepak Nibade
2b7e8a2c2a gpu: nvgpu: fix channel unbind sequence from TSG
We right now remove a channel from TSG list and disable all the channels in
TSG while removing a channel from TSG
With this sequence if any one channel in TSG is closed, rest of the channels
are set as timed out and cannot be used anymore

We need to fix this sequence as below to allow removing a channel from active
TSG so that rest of the channels can still be used

- disable all channels of TSG
- preempt TSG
- check if CTX_RELOAD is set if support is available
  if CTX_RELOAD is set on channel, it should be moved to some other channel
- check if FAULTED is set if support is available
- if NEXT is set on channel then it means channel is still active
  print out an error in this case for the time being until properly handled
- remove the channel from runlist
- remove channel from TSG list
- re-enable rest of the channels in TSG
- clean up the channel (same as regular channels)

Add below fifo operations to support checking channel status
g->ops.fifo.tsg_verify_status_ctx_reload
g->ops.fifo.tsg_verify_status_faulted

Define ops.fifo.tsg_verify_status_ctx_reload operation for gm20b/gp10b/gp106
as gm20b_fifo_tsg_verify_status_ctx_reload()
This API will check if channel to be released has CTX_RELOAD set, if yes
CTX_RELOAD needs to be moved to some other channel in TSG

Remove static from channel_gk20a_update_runlist() and export it

Bug 200327095

Change-Id: I0dd4be7c7e0b9b759389ec12c5a148a4b919d3e2
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1560637
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-09-15 12:48:21 -07:00
Deepak Nibade
460951ed09 gpu: nvgpu: fix TSG enable sequence
Due to a h/w bug in Maxwell and Pascal we first need to enable all channels with
NEXT and CTX_RELOAD set in a TSG, and then rest of the channels should be
enabled
Add this sequence to gk20a_tsg_enable()

Add new APIs to enable/disable scheduling of TSG runlist
gk20a_fifo_enable_tsg_sched()
gk20a_fifo_disble_tsg_sched()

Add new APIs to check if channel has NEXT or CTX_RELOAD set
gk20a_fifo_channel_status_is_next()
gk20a_fifo_channel_status_is_ctx_reload()

Bug 1739362

Change-Id: I4891cbd7f22ebc1e0bf32c52801002cdc259dbe1
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1560636
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-09-15 12:48:20 -07:00
Deepak Nibade
7d6d040531 gpu: nvgpu: support platform specific TSG enable/disable
Add platform specific operations to enable/disable a TSG and use them instead
of directly calling enable/disable APIs

For gm20b/gp106/gp10b we continue to use gk20a_enable_tsg() and
gk20a_disable_tsg() as platform specific operations

Bug 1739362

Change-Id: I2dd0f38c8303757e8c7a47d8da0e30a790e514f0
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1560635
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-09-15 12:48:20 -07:00
Alex Waterman
72f40c211e gpu: nvgpu: page align DMA allocs
Explicitly page align DMA memory allocations. Non-page aligned
DMA allocs can lead to nvgpu_mem structs that have a size that's
not page aligned. Those allocs, in some cases, can cause vGPU
maps to return an error.

More generally DMA allocs in Linux are never going to not be
page aligned both in size and address. So might as well page align
the alloc size to make code else where in the driver more simple.

To imlpement this an aligned_size field has been added to struct
nvgpu_mem. This field has the real page aligned size of the
allocation. The original size is still saved in size.

Change-Id: Ie08cfc4f39d5f97db84a54b8e314ad1fa53b72be
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1547902
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-09-12 15:54:46 -07:00
Terje Bergstrom
89772b03cb gpu: nvgpu: Move XVE debugfs code to Linux module
Move XVE debugfs initialization code to live under common/linux.

JIRA NVGPU-62

Change-Id: Ic6677511d249bc0a2455dde01db5b230afc70bb1
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1535133
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-09-12 13:00:19 -07:00
Deepak Nibade
41283a02ac gpu: nvgpu: fix UBSAN warning of signed integer overflow
Fix below warning reported by UBSAN by explicitly type casting both the
operators in multiplication as unsigned long

[   69.470802] UBSAN: Undefined behaviour in
drivers/gpu/../../../nvgpu/drivers/gpu/nvgpu/gk20a/gk20a_scale.c:60:49
[   69.485519] signed integer overflow:
[   69.489104] 2147483647 * 1000 cannot be represented in type 'int'
[   69.504424] Hardware name: quill (DT)
[   69.508088] Call trace:
[   69.510579] [<ffffff900809a600>] dump_backtrace+0x0/0x4f0
[   69.515987] [<ffffff900809ab18>] show_stack+0x28/0x38
[   69.521050] [<ffffff9008f0d8d8>] dump_stack+0x154/0x1c4
[   69.526291] [<ffffff9008fd1ee0>] ubsan_epilogue+0x18/0xb0
[   69.531720] [<ffffff9008fd321c>] handle_overflow+0x1c0/0x21c
[   69.537416] [<ffffff9008fd3334>] __ubsan_handle_mul_overflow+0x34/0x50
[   69.544410] [<ffffff9003c73368>] gk20a_scale_qos_notify+0x210/0x2f0 [nvgpu]
[   69.551415] [<ffffff9008170884>] __blocking_notifier_call_chain+0xec/0x240
[   69.558299] [<ffffff9008170a18>] blocking_notifier_call_chain+0x40/0x50
[   69.564928] [<ffffff900825dd18>] pm_qos_update_bounded_target+0x738/0x1038
[   69.571812] [<ffffff900825f4a0>] pm_qos_update_bounded_req+0x148/0x280
[   69.578348] [<ffffff9008263224>] pm_qos_bounded_write+0x484/0x990

Bug 200342586

Change-Id: I35ee59a95b2e3625fb42f256d2877558be9e51cf
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1557156
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Pritesh Raithatha <praithatha@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-09-12 11:05:55 -07:00
Terje Bergstrom
c37c9baae6 gpu: nvgpu: Move CDE code to Linux module
CDE is only used in Linux platforms, and the code is highly dependent
on Linux APIs. Move the common CDE code to Linux module and leave only
the chip specific parts to HAL.

Change-Id: I507fe7eceaf7607303dfdddcf438449a5f582ea7
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1554755
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-09-11 15:10:52 -07:00
Konsta Holtta
17451138cf gpu: nvgpu: hold ch ref when getting ch from fd
Fix a race condition in gk20a_get_channel_from_file() that returns a
channel pointer from an fd: take a reference to the channel before
putting the file ref back. Now the caller is responsible of releasing
the channel reference eventually.

Also document why dbg_session_channel_data has to hold a ref to the
channel file instead of just the channel: that might deadlock if the fds
were closed in "wrong" order.

Change-Id: I8e91b809f5f7b1cb0c1487bd955ad6d643727a53
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1549290
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-09-11 10:34:57 -07:00
Terje Bergstrom
bb457e675e gpu: nvgpu: Remove mapping of comptags to user VA
Remove the kernel feature to map compbit backing store areas to GPU VA.
The feature is not used, and the relevant user space code is getting
removed, too.

Change-Id: I94f8bb9145da872694fdc5d3eb3c1365b2f47945
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1547898
Reviewed-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-by: Sami Kiminki <skiminki@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
2017-09-07 20:07:16 -07:00
Sandeep Shinde
33f192b2f7 gpu: nvgpu: Add pd_max_batches sysfs node for gp10b
Add a new sysfs node pd_max_batches for setting max batches value in
NV_PGRAPH_PRI_PD_AB_DIST_CONFIG_1_MAX_BATCHES register which controls
max number of batches per alpha-beta transition stored in PD.

Bug 1927124

Change-Id: I2817f2d70dab348d8b0b8ba19bf1e9b9d23ca907
Signed-off-by: Sandeep Shinde <sashinde@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1544104
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
(cherry picked from commit aa4daddda2)
Reviewed-on: https://git-master.nvidia.com/r/1543355
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
GVS: Gerrit_Virtual_Submit
2017-09-07 20:07:13 -07:00
David Li
a199baede7 gpu: nvgpu: add NVGPU_SUBMIT_GPFIFO_FLAGS_RESCHEDULE_RUNLIST
NVGPU_SUBMIT_GPFIFO_FLAGS_RESCHEDULE_RUNLIST causes host to expire
current timeslice and reschedule from front of runlist.
This can be used with NVGPU_RUNLIST_INTERLEAVE_LEVEL_HIGH to make a
channel start sooner after submit rather than waiting for natural
timeslice expiration or block/finish of currently running channel.

Bug 1968813

Change-Id: I632e87c5f583a09ec8bf521dc73f595150abebb0
Signed-off-by: David Li <davli@nvidia.com>
Reviewed-on: http://git-master/r/#/c/1537198
Reviewed-on: https://git-master.nvidia.com/r/1537198
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-08-31 18:44:54 -07:00
Sunny He
cc64606a53 gpu: nvgpu: debugfs code to dump HAL functions
Prints addresses of device-specific HAL functions to debugfs file
hal/gops.

The list of functions is produced by dumping the contents of the
gpu_ops substruct of the gk20a struct. This interface makes the
assumption that there are only function pointers in gpu_ops.

Companion Python script nvgpu_debug_hal.py analyzes gk20a.h to
determine operation counts and prettyify debugfs interface's
output.

Jira NVGPU-107

Change-Id: I0910e86638d144979e8630bbc5b330bccfd3ad94
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1542990
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-08-24 09:34:52 -07:00
Debarshi Dutta
3fa47b877d gpu: nvgpu: Replace kref for refcounting in nvgpu
- added wrapper struct nvgpu_ref over nvgpu_atomic_t
- added nvgpu_ref_* APIs to access the above struct

JIRA NVGPU-140

Change-Id: Id47f897995dd4721751f7610b6d4d4fbfe4d6b9a
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1540899
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
2017-08-24 01:10:37 -07:00
Terje Bergstrom
bc87e8989d gpu: nvgpu: Remove support for old kernel version
Remove support for pre-4.4 kernels. This allows deleting the checks
for kernel version, and usage of linux/version.h.

Change-Id: I4d6cb30512ea164d27549f4f4d096e5931bb1379
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1543499
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-08-22 12:44:41 -07:00
Terje Bergstrom
af9e70f4d1 gpu: nvgpu: Copy vbios_min_version to struct gk20a
Accessing vbios_min_version in gk20a_platform creates an extra
dependency to Linux. Copy it to struct gk20a at driver
initialization.

Change-Id: I9ff5dbeb1fecc6dc44a62f7affc24fd52c2bab26
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1542837
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-08-22 08:55:13 -07:00
Debarshi Dutta
81868a187f gpu: nvgpu: Nvgpu abstraction for linux barriers.
construct wrapper nvgpu_* methods to replace
mb,rmb,wmb,smp_mb,smp_rmb,smp_wmb,read_barrier_depends and
smp_read_barrier_depends.

NVGPU-122

Change-Id: I8d24dd70fef5cb0fadaacc15f3ab11531667a0df
Signed-off-by: Debarshi <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1541199
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Sourab Gupta <sourabg@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
2017-08-22 03:53:51 -07:00
Sunny He
5f010177de gpu: nvgpu: Reorg pmu HAL initialization
Reorganize HAL initialization to remove inheritance and construct
the gpu_ops struct at compile time. This patch only covers the
pmu sub-module of the gpu_ops struct.

Perform HAL function assignments in hal_gxxxx.c through the
population of a chip-specific copy of gpu_ops.

Jira NVGPU-74

Change-Id: I8839ac99e87153637005e23b3013237f57275c54
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1530982
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-08-21 13:06:07 -07:00
Sunny He
b50b379c19 gpu: nvgpu: Move non-fp pmu members from gpu_ops
Move non-function pointer members out of the pmu and pmu_ver
substructs of gpu_ops. Ideally gpu_ops will have only function
ponters, better matching its intended purpose and improving
readability.

 - g.ops.pmu_ver.cmd_id_zbc_table_update has been changed to
    g.pmu_ver_cmd_id_zbc_table_update
 - g.ops.pmu.lspmuwprinitdone has been changed to
    g.pmu_lsf_pmu_wpr_init_done
 - g.ops.pmu.lsfloadedfalconid has been changed to
    g.pmu_lsf_loaded_falcon_id

Boolean flags have been implemented using the enabled.h API
 - g.ops.pmu_ver.is_pmu_zbc_save_supported moved to
    common flag NVGPU_PMU_ZBC_SAVE
 - g.ops.pmu.fecsbootstrapdone moved to
    common flag NVGPU_PMU_FECS_BOOTSTRAP_DONE

Jira NVGPU-74

Change-Id: I08fb20f8f382277f2c579f06d561914c000ea6e0
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1530981
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-08-21 13:06:04 -07:00
Peter Daifuku
a0be159e69 gpu: nvgpu: channel_worker_enqueue: warn to info
gk20a_channel_worker_enqueue currently WARNS if the call to gk20a_channel_get
fails. But in fact a failure here is legitimate, in particular if the channel
is in the process of being freed, and the referenceable flag changed between
the time the caller grabbed a reference to the channel and the time this
function tries to grab its own reference.

Instead, turn the warning into an info message.

Bug 200329971

Change-Id: Iab4ca6b32bb98f42f05b12b7971538db962b9795
Signed-off-by: Peter Daifuku <pdaifuku@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1540458
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-08-17 16:45:05 -07:00
Debarshi Dutta
98186ec2c2 gpu: nvgpu: Add wrapper over atomic_t and atomic64_t
- added wrapper structs nvgpu_atomic_t and nvgpu_atomic64_t over
  atomic_t and atomic64_t
- added nvgpu_atomic_* and nvgpu_atomic64_* APIs to access the above
  wrappers.

JIRA NVGPU-121

Change-Id: I61667bb0a84c2fc475365abb79bffb42b8b4786a
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1533044
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
GVS: Gerrit_Virtual_Submit
2017-08-17 14:26:47 -07:00
Alex Waterman
61ef5a5874 gpu: nvgpu: Deterministic submit fix
Fix some simple pointer arithmetic errors in the deterministic submit
path. The lockless allocator was doing a subtraction to determine the
correct offset of the element to free. However, this subtraction was
using the base address and a numeric offset - not a pointer offset. Thus
the difference computed was in bytes not in elements of the block size.

The fix is simple: just divide by the block size.

Also this modifies the debugging statement a bit so that a bit more
information is printed at more useful times.

Lastly, a pointer to numeric cast was fixed in the fence code.

Change-Id: I514724205f1b73805b21e979481a13ac689f3482
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1538905
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Aingara Paramakuru <aparamakuru@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-08-16 11:04:20 -07:00
Alex Waterman
ff38ab4dcd gpu: nvgpu: Disable rd_coalesce for all chips
Disable read coalescing for all chips.

Bug 200314091

Change-Id: Iaa3f58f94369ae1edae0620083eca4594be730fd
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1518308
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-08-16 03:56:35 -07:00
Sunny He
959c02d675 gpu: nvgpu: Reorg mm HAL initialization
Reorganize HAL initialization to remove inheritance and construct
the gpu_ops struct at compile time. This patch only covers the
mm sub-module of the gpu_ops struct.

Perform HAL function assignments in hal_gxxxx.c through the
population of a chip-specific copy of gpu_ops.

Jira NVGPU-74

Change-Id: Ieb87a62f047510e51c52e6563d8e3fd5a65b5f28
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1537753
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-08-14 15:55:19 -07:00
Sunny He
d5a5fb3563 gpu: nvgpu: remove kind map inheritance
Currently, the kind map initialization inherits from earlier
chips' kind map definitions and simply adds on newly supported
features. This is dangerous as changes in older architectures
may affect newer architectures in unpredictable ways.

This patch removes inheritance between subsequent architectures'
kind map initializations.

Jira NVGPU-74

Change-Id: Ie5c33f613db7ba109564b4cb2312e47ee3332afc
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1537745
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-08-14 15:55:13 -07:00
Sai Gurrappadi
0e81ecf021 gpu: nvgpu: gk20a: Fix up qos/sysfs limits
Userspace sysfs limits devfreq->{min|max}_freq and pmqos min/max limits
are treated as the same priority. In case these two ranges don't intersect
prefer the max_freq clamp over the min_freq clamp.

This is done in order to ensure any thermal throttling isn't bypassed by a
userspace sysfs write. MaxQ/MaxP configurations only care about the Fmax
limit and not necessarily Fmin, so preferring max_freq over min_freq is
consistent with what is required.

Bug 1933216

Change-Id: Ie8685306f10dee0f8d340c4b8e0eae533dbfc6ac
Signed-off-by: Sai Gurrappadi <sgurrappadi@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1535937
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: Aaron Huang <aaronh@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svc-mobile-misra <svc-mobile-misra@nvidia.com>
Reviewed-by: Deepak Nibade <dnibade@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-08-11 15:46:09 -07:00
Sunny He
5248d6b4b5 Revert "gpu: nvgpu: remove kind map inheritance"
Conflicts with gv100 changes

This reverts commit 83e49b3918.

Change-Id: Ic5720fd3c214fa306645fb628a28ecf7d6d0c26d
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1537673
Reviewed-by: Shu Zhong <shuz@nvidia.com>
Tested-by: Shu Zhong <shuz@nvidia.com>
2017-08-11 14:57:42 -07:00
Sunny He
f8399cfa55 Revert "gpu: nvgpu: Reorg mm HAL initialization"
Conflicts with gv100 changes

This reverts commit 8d63cd3995.

Change-Id: Ie2f88d281b2b87a9a794d79164a61c4d883626b7
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1537668
Reviewed-by: Shu Zhong <shuz@nvidia.com>
Tested-by: Shu Zhong <shuz@nvidia.com>
2017-08-11 14:57:15 -07:00
Sunny He
8d63cd3995 gpu: nvgpu: Reorg mm HAL initialization
Reorganize HAL initialization to remove inheritance and construct
the gpu_ops struct at compile time. This patch only covers the
mm sub-module of the gpu_ops struct.

Perform HAL function assignments in hal_gxxxx.c through the
population of a chip-specific copy of gpu_ops.

Jira NVGPU-74

Change-Id: I289284e6e528fc7951c959c8765ccf9349eec33b
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1533351
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-08-11 14:16:52 -07:00
Sunny He
83e49b3918 gpu: nvgpu: remove kind map inheritance
Currently, the kind map initialization inherits from earlier
chips' kind map definitions and simply adds on newly supported
features. This is dangerous as changes in older architectures
may affect newer architectures in unpredictable ways.

This patch removes inheritance between subsequent architectures'
kind map initializations.

Jira NVGPU-74

Change-Id: I21951c0920f69ced73088cb6670ca1894159c7b9
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1533349
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-08-11 14:16:46 -07:00
David Nieto
eb9864d92d gpu: nvgpu: PG503 support
Adds basic PG503 support allowing devinit to complete.

JIRA: EVLR-1693

Change-Id: Ice8a9ba18c8bba11f6bc174ba2c2d8802a738706
Signed-off-by: David Nieto <dmartineznie@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1532746
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Tested-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-08-11 11:07:17 -07:00
Richard Zhao
3a83185003 gpu: nvgpu: fix pbdma_id searching when init engine info
The searching loop should break once find a match.

Jira VFND-3797

Change-Id: I43ef553a3e90afb00ee9a4df7d269b7c6616b18e
Signed-off-by: Richard Zhao <rizhao@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1535304
Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-08-11 09:28:16 -07:00
Alex Waterman
2492142224 gpu: nvgpu: Increase small page aperture
Increase the small page aperture to 56GB to facilitate easier
fixed address mapping for userspace (primarily CUDA).

Bug 200320732

Change-Id: I1f0aaa4f28c8a294cc880b35f26942b562396b48
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1502432
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-08-07 00:24:29 -07:00
Alex Waterman
1da69dd8b2 gpu: nvgpu: Remove mm.get_iova_addr
Remove the mm.get_iova_addr() HAL and replace it with a new HAL
called mm.gpu_phys_addr(). This new HAL provides the real phys
address that should be passed to the GPU from a physical address
obtained from a scatter list. It also provides a mechanism by
which the HAL code can add extra bits to a GPU physical address
based on the attributes passed in. This is necessary during GMMU
page table programming.

Also remove the flags argument from the various address functions.
This flag was used for adding an IO coherence bit to the GPU
physical address which is not supported.

JIRA NVGPU-30

Change-Id: I69af5b1c6bd905c4077c26c098fac101c6b41a33
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1530864
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-08-04 14:54:32 -07:00
Terje Bergstrom
192cf8c1f8 gpu: nvgpu: Pass struct gk20a to gk20a_wait_for_idle()
Pass struct gk20a pointer instead of struct device to
gk20a_wait_for_idle(). The code is not Linux specific and does not
need pointer to struct device.

Change-Id: I2cafd6c7db019c9de76b6e68a1ae73f0b4cea37d
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1533173
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alex Waterman <alexw@nvidia.com>
2017-08-04 13:44:19 -07:00
Terje Bergstrom
a0a3a6323a gpu: nvgpu: Pass gk20a to XVE init
Pass gk20a pointer instead of struct device pointer to XVE init.

Change-Id: I5f5efc1d09430a45a49321830b1d7f17c7a57e2f
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1528264
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Alex Waterman <alexw@nvidia.com>
GVS: Gerrit_Virtual_Submit
2017-08-04 09:24:02 -07:00
Terje Bergstrom
972c42fddf gpu: nvgpu: Move debugfs fields to os_linux
Move all Linux specific debugfs dentry fields to struct
nvgpu_os_linux.

JIRA NVGPU-62

Change-Id: I615620005f5d042943dd2e478c1629bcc912c550
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1528263
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-08-04 09:24:02 -07:00
Terje Bergstrom
b8efd9d045 gpu: nvgpu: Make LTC disabling common code
Refactor the sync_debugfs LTC HAL op so that the logic to enable
or disable LTC goes to common code nvgpu_ltc_sync_enabled() and
the LTC HAL set_enabled only performs the hardware register access.

Create a new common function nvgpu_init_ltc_support() to initialize
the LTC software variable, and move hardware initialization of LTC to
be called from it.

JIRA NVGPU-62

Change-Id: Ib1cf4f5b83ca3dac08407464ed56a732e0a33923
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1528262
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-08-04 09:23:56 -07:00
Terje Bergstrom
e1df72771b gpu: nvgpu: Move isr related fields from gk20a
Move fields in struct gk20a related to interrupt handling into
Linux specific nvgpu_os_linux. At the same time move the counter
logic from function in HAL into Linux specific code, and two Linux
specific power management functions from generic gk20a.c to Linux
specific module.c.

JIRA NVGPU-123

Change-Id: I0a08fd2e81297c8dff7a85c263ded928496c4de0
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1528177
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Sourab Gupta <sourabg@nvidia.com>
GVS: Gerrit_Virtual_Submit
2017-08-03 08:44:58 -07:00
Seema Khowala
04470a984d gpu: nvgpu: support multiple sm for t19x
Allocate memory for sm_to_cluster to support multiple sm

Bug 1951026

Change-Id: I48256f097fbc95e4432950a7e8de62577819a025
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1515778
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2017-07-28 05:14:03 -07:00
Peter Daifuku
02acac71b3 gpu: nvgpu: avoid possible ovrflw in dmabuf check
In gk20a_vm_map_buffer, when checking dmabuf size, avoid possible
overflow of buffer offset + buffer size

Bug 1793926

Change-Id: Iaa85bbd2942546015a233f34388309c6ba01412c
Signed-off-by: Peter Daifuku <pdaifuku@nvidia.com>
Reviewed-on: http://git-master/r/1488051
(cherry picked from commit 62346ede6c0863d36dc5d91527647130a13eff53)
Reviewed-on: http://git-master/r/1501696
(cherry picked from commit 745c273ac80fad14f019b7c59bb797c4e22f4781)
Reviewed-on: https://git-master.nvidia.com/r/1528182
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Vladislav Buzov <vbuzov@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-07-27 21:41:21 -07:00
Sunny He
f0593bb43f gpu: nvgpu: Reorg pramin HAL initialization
Reorganize HAL initialization to remove inheritance and construct
the gpu_ops struct at compile time. This patch only covers the
pramin sub-module of the gpu_ops struct.

Perform HAL function assignments in hal_gxxxx.c through the
population of a chip-specific copy of gpu_ops.

Jira NVGPU-74

Change-Id: I0c0aecfb8f5ea436ef353b874f5e36ff24ebd130
Signed-off-by: Sunny He <suhe@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1527421
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2017-07-27 16:34:45 -07:00