Commit Graph

1407 Commits

Author SHA1 Message Date
Konsta Holtta
a971510737 gpu: nvgpu: add nvgpu_mem_rd32_pair
Sometimes longer 64-bit values are composed of pairs of low and high
parts in a same nvgpu_mem. Add a function to read the two and to do the
bit ops given the low and high word offsets.

Jira NVGPU-886

Change-Id: I4fe698d7c5a891ef5740ea0fb31549195e2e6c43
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1928206
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-30 15:35:16 -07:00
Amurthyreddy
9aa74d5f86 gpu: nvgpu: MISRA 10.4 boolean fixes
MISRA rule 10.4 doesn't allow arithmetic conversions on operands of
different essential type category.

Fix violations where an arithmetic conversion is performed on boolean
and non-boolean types.

JIRA NVGPU-994

Change-Id: I2af9937678462b632bb6ec6178e10d02104fc3bc
Signed-off-by: Amurthyreddy <amurthyreddy@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1832337
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-30 15:33:58 -07:00
Konsta Holtta
b08c613402 gpu: nvgpu: make gr_ctx a pointer in tsg
Remove a dependency to a graphics type in tsg header by adding a pointer
indirection.

Jira NVGPU-967
Jira NVGPU-1149

Change-Id: I9177e6eedf08bfe4a3b981b67fa8d4d734f9e50f
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1822023
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-30 05:54:10 -07:00
Alex Waterman
aee5511bc8 gpu: nvgpu: Update pd_cache to handle 64K pages
Update the PD cache code to handle 64KB pages. To do this the
number of partial/full lists is expanded for when we have 64K
pages. Currently we only explicitly support 4K and 64K page
sizes. Other pages sizes (16K for example) will fail compilation
during preprocessing.

This change also cleans up the definitions for some of the
internal structs. They have been moved into pd_cache.c since
they are not used outside of pd_cache.c.

This allows the following functions to be removed from the global
context:

  __nvgpu_pd_cache_alloc_direct()
  __nvgpu_pd_cache_free_direct()

They have been replaced by calls to nvgpu_pd_{alloc,free}().

The nvgpu_pd_mem_entry alloc_map also had to be expanded to a
real bitmap. 32 or 64 bits is not sufficient for packing 256
byte PDs into a 64K page (there's 256 PDs per nvgpu_pd_mem_entry
in that case). To prevent doing too many find_first_zero
operations on the bitmap an 'allocs' field was also added which
tracks how many allocs are done. We can use this instead of
comparing a mask against the bitmap to determine if an
nvgpu_pd_mem_entry is full.

Note: there's still a limitation with the TLB invalidate code:
it simply assumes an nvgpu_mem is a 1 to 1 with a PDB. This means
we can't invalidate a PDB allocated at an offset greater than 0
in a nvgpu_pd_mem_entry. This in turn means we must always use a
full page size for a context's PDB.

Bug 1977822

Change-Id: I6a7a3a95b7c902bc6487cba05fde58fbc4a25da5
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1718755
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-29 21:56:59 -07:00
Alex Waterman
c64f9432b1 gpu: nvgpu: Fix comment in priv_cmd_buf allocation
Update the comment to fix obvious issues and describe the
new allocation logic.

Bug 2327792

Change-Id: Ica0dd4159467e3023cc487a2bf9f525db3ad76e6
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1831096
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-29 17:06:45 -07:00
Alex Waterman
b9ec592f1d gpu: nvgpu: Make priv_cmd_buf honor num_in_flight jobs
If num_in_flight jobs is set use that to determine the proper
size of the priv_cmd_buf. If num_in_flight is not set then use
the original logic: the priv_cmd_buf is sized based on a worst
case assumption for the GPFIFO.

Also clean up MISRA issues.

Bug 2327792

Change-Id: Ie192caeb6cc48fdcac57e5cbb71c534aeaf46011
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1831095
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-29 17:06:41 -07:00
Alex Waterman
05ec7b80eb gpu: nvgpu: Use deterministic flag to decide pre-alloc
Instead of using num_inflight_jobs to determine whether to pre-alloc
resources for a channel use the c->deterministic flag and the
number of inflight jobs field. Non-determinsitic channels do not
require pre-alloced resources and deterministic channels with 0
in flight jobs (i.e no kernel job tracking, AKA fast path sumits)
also do not require pre-alloced resources.

Bug 2327792

Change-Id: I7e8eb0478c22e005ca2c46c555415afa0ded0be1
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1850123
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svc-misra-checker <svc-misra-checker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-29 17:06:37 -07:00
Amulya
3e6a445310 nvgpu: common: MISRA 10.1 boolean fixes
Fix violations where a variable of type non-boolean is used as a
boolean in gpu/nvgpu/common.

JIRA NVGPU-646

Change-Id: I64e96e02e9a3d5d5604c4fa52460e0415f484d75
Signed-off-by: Amulya <Amurthyreddy@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1807128
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svc-misra-checker <svc-misra-checker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Tested-by: Amulya Murthyreddy <amurthyreddy@nvidia.com>
Reviewed-by: Adeel Raza <araza@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-29 14:44:38 -07:00
Konsta Holtta
99b1c6dcdf gpu: nvgpu: support usermode submit buffers
Import userd and gpfifo buffers from userspace if provided via
NVGPU_IOCTL_CHANNEL_ALLOC_GPFIFO_EX. Also supply the work submit token
(i.e., the hw channel id) to userspace.

To keep the buffers alive, store their dmabuf and attachment/sgt handles
in nvgpu_channel_linux. Our nvgpu_mem doesn't provide such data for
buffers that are mainly in kernel use. The buffers are freed via a new
API in the os_channel interface.

Fix a bug in gk20a_channel_free_usermode_buffers: also unmap the
usermode gpfifo buffer.

Bug 200145225

Change-Id: I8416af7085c91b044ac8ccd9faa38e2a6d0c3946
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1795821
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-29 08:04:43 -07:00
smadhavan
757f028320 gpu: nvgpu: MISRA Rule 21.2 header guard fixes
MISRA rule 21.2 doesn't allow the use of macro names which start with
an underscore. These leading underscores are to be removed from the
macro names. This patch will fix such violations by renaming them
to follow the convention, 'NVGPU_PARENT-DIR_HEADER-NAME' when there
is no keyword repetition between file name and directory or
'NVGPU_HEADER-NAME' when there is repetition.

Deviate from the convention mentioned earlier in the case of clk.h.
There are two files with name clk.h. One under include/nvgpu directory
and other on in clk/ directory. Both of them will have header guards
with name NVGPU_CLK_H under the convention. Thus clk/clk.h will have
header guard with name NVGPU_CLK_CLK_H and include/nvgpu/clk will have
header guard with name NVGPU_INCLUDE_CLK_H

JIRA NVGPU-1028

Change-Id: I386d39649f39a98019a9e06bc65c8b623a43629b
Signed-off-by: smadhavan <smadhavan@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1923539
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-29 03:57:01 -07:00
Scott Long
a5d5454c8b gpu: nvgpu: MISRA 21.15 fixes to bios code
MISRA Rule 21.15 prohibits use of memcpy() with incompatible ptrs
to qualified/unqualified types.

To circumvent this issue we've introduced a new MISRA-compliant
nvgpu_memcpy() function.

This change switches all offending uses of memcpy() in bios.c over to
use nvgpu_memcpy() with appropriate casts applied.

JIRA NVGPU-849

Change-Id: I218068dd1f1f2c978ea8ab2ba230b3e5fb3da171
Signed-off-by: Scott Long <scottl@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1936149
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svc-misra-checker <svc-misra-checker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-26 17:29:55 -07:00
Amurthyreddy
89660dbd62 gpu: nvgpu: MISRA 14.4 bitwise operation as boolean
MISRA rule 14.4 doesn't allow the usage of integer types as booleans
in the controlling expression of an if statement or an iteration
statement.

Fix violations where the result of a bitwise operation is used as a
boolean in the controlling expression of if and loop statements.

JIRA NVGPU-1020

Change-Id: If910150072c3dd67c31fe9819c3a9e738fd3c1c6
Signed-off-by: Amurthyreddy <amurthyreddy@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1932389
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svc-misra-checker <svc-misra-checker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Adeel Raza <araza@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-26 10:06:55 -07:00
Debarshi Dutta
6fe9bb835b gpu: nvgpu: access channel_sync via public API
struct nvgpu_channel_sync is moved to a private header i.e.
channel_sync_priv.h present in common/sync/. All accesses to callback
functions inside the struct nvgpu_channel_sync in NVGPU driver is replaced by
the public channel_sync specific APIs.

Jira NVGPU-1093

Change-Id: I52d57b3d458993203a3ac6b160fb569effbe5a66
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1929783
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-26 02:12:23 -07:00
Debarshi Dutta
6456cbec85 gpu: nvgpu: separate semaphore function definitions to separate unit
Add the following public APIs.

nvgpu_channel_sync_to_semaphore
nvgpu_channel_sync_semaphore_create

struct nvgpu_channel_sync_semaphore and semaphore specific static
implementations of the channel_sync callbacks as well as
definitions of the public APIs are moved to a
separate execution unit i.e. channel_sync_semaphore.c.

Jira NVGPU-1093

Change-Id: I32c62a75ef999e8f3047e2a593f77d32cbde5646
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1929781
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-26 02:12:14 -07:00
Debarshi Dutta
715d35e137 gpu: nvgpu: separate syncpoint function definitions to separate unit
Add the following syncpt specific APIs

nvgpu_channel_sync_get_syncpt_id
nvgpu_channel_sync_get_syncpt_address
nvgpu_channel_sync_wait_syncpt
nvgpu_channel_sync_to_syncpt
nvgpu_channel_sync_syncpt_create

Definition of struct nvgpu_channel_sync_syncpt and syncpoint
specific static implementations of the channel_sync callbacks
as well as definitions of the public APIs are moved to a
separate execution unit i.e. channel_sync_syncpt.c

Jira NVGPU-1093

Change-Id: Ib0163c6b9bc6dfc2ab2a2b7a5fa5027be13316e2
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1929780
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-26 02:12:05 -07:00
Debarshi Dutta
0eac7ed9a2 gpu: nvgpu: add public APIs for nvgpu_channel_sync
Add public APIs for accessing callback functions of struct
nvgpu_channel_sync. The following public functions are added.

nvgpu_channel_sync_wait_fence_fd
nvgpu_channel_sync_incr
nvgpu_channel_sync_incr_user
nvgpu_channel_sync_set_min_eq_max
nvgpu_channel_sync_set_safe_state

Jira NVGPU-1093

Change-Id: I1ac149f340af77c7b71dbf7fdab78931bb59203b
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1929779
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-26 02:11:56 -07:00
Deepak Nibade
7ed3d0dcf4 gpu: nvgpu: tu104: support SLCG/BLCG
Generate gating register list for Turing SLCG/BLCG in
common/clock_gating/tu104_gating_reglist.c

Set all the gops.clock_gating HALs

Jira NVGPUT-108
Bug 200456693

Change-Id: Ie7e3e6951b1eea0c48a25db93d391b7a82df5fd9
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1919938
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-26 02:11:08 -07:00
Mahantesh Kumbar
c032207351 gpu: nvgpu: PMU PSTATE init wait for pmu_ready
-On Turing, LS PMU RTOS is bootstrapped by SEC2 RTOS,
 so PMU needs some time to boot & get ready to process
 request as boot happened bit late in flow.
-So issue here is, code execution reaches function
 gk20a_init_pstate_pmu_support() to send boardobj
 commands to PMU before PMU ready to accept.
-As a result, this cause command request failure
 as PMU is not ready yet due to falcon clock is
 slower compared CPU clock.

-To fix issue, waiting for pmu_ready flag to become
 true before staring PSTATE requests to PMU.
-pmu_ready flag will be set to true as soon as
 INIT message received from PMU RTOS.

-Pre Turing-PMU RTOS will be ready much ahead of
 reaching this point.

JIRA NVGPU-1150

Change-Id: I4fb5d4ca6dabc22fac34ae32880d7a1f3164e1b7
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1925240
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-25 13:43:22 -07:00
Scott Long
c08b987db3 gpu: nvgpu: add MISRA-compliant string ops
Add nvgpu_memcpy/nvgpu_memcmp which are MISRA-compliant versions
(Rule 21.15) of memcpy/memcmp.

Also convert some clk/gr calls over to use the new routines;
all of the remaining calls will be converted in subsequent patches.

JIRA NVGPU-849

Change-Id: Ib3a602cd08886764ba9a50285462a8b07bfb18ba
Signed-off-by: Scott Long <scottl@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1919470
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-25 12:53:40 -07:00
Adeel Raza
dc37ca4559 gpu: nvgpu: MISRA fixes for composite expressions
MISRA rules 10.6, 10.7, and 10.8 prevent mixing of types in composite
expressions. Resolve these violations by casting variables/constants to
the appropriate types.

Jira NVGPU-850
Jira NVGPU-853
Jira NVGPU-851

Change-Id: If6db312187211bc428cf465929082118565dacf4
Signed-off-by: Adeel Raza <araza@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1931156
GVS: Gerrit_Virtual_Submit
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-25 11:13:38 -07:00
aalex
80d03f34f7 gpu: nvgpu: Fix IPA to PA translation
Background:
In Hypervisor mode dGPU device is configured in pass through mode for
the Guest (QNX/Linux). GMMU programming is handled by the guest which
converts a mapped buffer's GVA into SGLes in IPA (Intermediate/Guest
Physical address) which is then translated into PA (Acutual Physical
address) and programs the GMMU PTEes with correct GVA to PA mapping.
Incase of the vgpu this work is delegated to the RM server which takes care
of the GMMU programming and IPA to PA conversion.

Problem:
The current GMMU mapping logic in the guest assumes that PA range is
continuous over a given IPA range. Hence, it doesn't account for holes being
present in the PA range. But this is not the case, a continous IPA range
can be mapped to dis-contiguous PA ranges. In this situation the mapping
logic sets up GMMU PTEes ignoring the holes in physical memory and
creates GVA => PA mapping which intrudes into the PA ranges which are
reserved. This results in memory being corrupted.

This change takes into account holes being present in a given PA range and
for a  given IPA range it also identifies the discontiguous PA ranges and
sets up the PTE's appropriately.

Bug  200451447
Jira VQRM-5069

Change-Id: I354d984f6c44482e4576a173fce1e90ab52283ac
Signed-off-by: aalex <aalex@nvidia.com>
Signed-off-by: Antony Clince Alex <aalex@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1850972
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-24 23:16:20 -07:00
Debarshi Dutta
6c8be7cfe2 gpu: nvgpu: move header location of gk20a.h
Change path corresponding to gk20a.h to <nvgpu/gk20a.h> corresponding
to files in the following directories.

gk20a/
vgpu/
gv100/
tu104/
common/bus/
common/fb/
common/ltc/
common/mc/
common/perf/

Jira NVGPU-597

Change-Id: I7b4f5e5ea3d13a4d1810c5db35fbc26fe5da443e
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1846826
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-24 23:16:10 -07:00
Amurthyreddy
f8ce19f879 gpu: nvgpu: MISRA 14.4 Function pointer as boolean
MISRA rule-14.4 doesn't allow the usage of function pointers & integer
types as booleans in the controlling expression of an if statement or
an iteration statement.

Fix violations where a function pointer or a function whose return
value is an integer, is used as a boolean in the controlling expression
of if and loop statements.

JIRA NVGPU-1021

Change-Id: Ic5336268394ba4396ce80744c25930d2fb44dc42
Signed-off-by: Amurthyreddy <amurthyreddy@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1932147
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-24 17:01:39 -07:00
Amurthyreddy
88d21daedb gpu: nvgpu: MISRA 14.4 err/ret/status as boolean
MISRA rule 14.4 doesn't allow the usage of integer types as booleans
in the controlling expression of an if statement or an iteration
statement.

Fix violations where the integer variables err, ret, status are used
as booleans in the controlling expression of if and loop statements.

JIRA NVGPU-1019

Change-Id: Ia2ec5f1db3c7a1884efe5ba7b8b4d9ebbd021734
Signed-off-by: Amurthyreddy <amurthyreddy@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1921373
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-24 17:00:11 -07:00
Terje Bergstrom
bc379d5eed gpu: nvgpu: Split L2 interrupt handling to MC and L2
L2 interrupt is processed by first reading from MC which L2 triggered
the interrupt and then calling a function per L2 slice to get the
details. Move the outer loop to MC unit, and the inner loop and L2
accesses to LTC unit.

JIRA NVGPU-954

Change-Id: I69b7bb82e4574b0519cdcd73b94d7d3e3fa6ef9e
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1851328
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-24 17:00:01 -07:00
Amurthyreddy
d522a2ddfc nvgpu: gk20a: MISRA 10.1 boolean fixes
Fix violations where a variable of type non-boolean is used as
boolean in gpu/nvgpu/gk20a.

JIRA NVGPU-646

Change-Id: Id02068c77f9385adb82c27ef1994a3f88499de48
Signed-off-by: Amurthyreddy <amurthyreddy@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1829584
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-24 16:59:39 -07:00
Deepak Nibade
1b2a0833e0 gpu: nvgpu: add separate unit for debugger
Rename gk20a/dbg_gpu_gk20a.c to common/debugger.c and make it a
separate common unit
Also rename gk20a/dbg_gpu_gk20a.h to include/nvgpu/debugger.h

We had two different HALs for debugger - gops.debugger and
gops.dbg_session_ops
Combine them into one single HAL gops.debugger and remove
gops.dbg_session_ops

Rename all exported APIs from debugger.h to be in the form of
nvgpu_*()

Jira NVGPU-1013

Change-Id: I136dc7786e3b2065921eb03b99f16049212f3cd2
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1920075
Reviewed-by: Sachin Jadhav <sachinj@nvidia.com>
Tested-by: Sachin Jadhav <sachinj@nvidia.com>
2018-10-24 00:30:19 -07:00
Sagar Kamble
3030707936 gpu: nvgpu: init auto objects for MISRA 9.1
Address MISRA Rule 9.1 violation: The value of an object with automatic
storage duration shall not be read before it has been set.

JIRA NVGPU-881

Change-Id: I63fde303b0a3e05f16b9ce518684a37c774f0f43
Signed-off-by: Sagar Kamble <skamble@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1929824
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-23 15:44:50 -07:00
Sagar Kamble
e67bb65025 gpu: nvgpu: update macro defines for MISRA 27.9
Address MISRA Rule 20.7 violation: Macro parameter expands into an
expression without being wrapped by parentheses.

Some of the exception the coverity is not able to catch are:
1. Macro parameters passed as parameter to another macro.
   i.e NVGPU_ACCESS_ONCE. Fixing these by additional parantheses.
2. Macro parameter used as type. i.e. type parameter in container_of.

While at it, update copyright date rage for list.h and types.h.

JIRA NVGPU-841

Change-Id: I4b793981d671069289720e8c041bad8125961c0c
Signed-off-by: Sagar Kamble <skamble@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1929823
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-23 15:44:41 -07:00
tkudav
745e720089 gpu: nvgpu: gv100: Update PMU ucode version
The GV100 PMU ucode needs to be updated to add support
for PS3.5 Clock Programming boardobj.
The entrysize in GV100 PMU ucode is increased to match
that on Turing R400 PMU ucode.

JIRA NVGPU-1153

Change-Id: Ied3163522bf4e124849517e90bfa42fe4b320a96
Signed-off-by: tkudav <tkudav@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1918174
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-22 00:45:03 -07:00
Kyle Guo
69b46a6174 gpu: nvgpu: fix timeout check logic in sync-unmap
Fixed timeout checking flow in sync-unmap where timeout message could
be printed while there is not a timeout.

The original codeflow doesn't check refcount of a mapped_buffer again
after 10ms waiting. So if the buffer is released during the last
round of waiting, the timeout message will still be printed. The new
codeflow combines refcount and timeout checking. So that there won't
be an inconsistency between the two.

Bug 200434475

Change-Id: Id0a0aebcb24906a1ec7113e669227788e729564b
Signed-off-by: Kyle Guo <kyleg@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1930236
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-21 12:56:01 -07:00
Terje Bergstrom
7dc15d6d33 gpu: nvgpu: Move boardobj to common
Move boardobj unit to live under common. It's common code. Also moves
the header files to include/nvgpu/ to indicate that they're meant to
be called from outside boardobj unit.

JIRA NVGPU-596

Change-Id: I57758371c47083e3f666e0cc6d05c48c6d070529
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1850419
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-19 17:24:58 -07:00
Konsta Holtta
e0c8a16c8d gpu: nvgpu: Add CHANNEL_SETUP_BIND IOCTL
For a long time now, the ALLOC_GPFIFO_EX channel IOCTL has done much
more than just gpfifo allocation, and its signature does not match
support that's needed soon. Add a new one called SETUP_BIND to hopefully
cover our future needs and deprecate ALLOC_GPFIFO_EX.

Change nvgpu internals to match this new naming as well.

Bug 200145225

Change-Id: I766f9283a064e140656f6004b2b766db70bd6cad
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1835186
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-19 17:24:49 -07:00
Nicolas Benech
89125cb4f5 gpu: nvgpu: pramin: add error checking for SGLs
If the total size of SGLs is lower than the size to copy,
we will reach the end of the list so the sgl var will become NULL,
and calling nvgpu_sgt_get_length will cause a null pointer dereference.
This change will cause a BUG() which should be clearer than a NULL
pointer dereference. There is no easy way to add more advanced error
checking and handling, and an SGL bug would most likely be linked to
another bug in the OS or OS layer.

JIRA NVGPU-1279

Change-Id: Ide83f2b91ecae25f3a0f3202febfb115110315d7
Signed-off-by: Nicolas Benech <nbenech@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1923706
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-16 23:41:08 -07:00
Mahantesh Kumbar
ec2b3a748f gpu: nvgpu: load dbg mem_unlock bin for dbg board
-Renamed mem_unlock.bin to mem_unlock_dbg.bin
 for MEM_UNLOCK_DBG_BIN define to load debug bin
 for INT board based on debug signal
 SCP_CTL_STAT_DEBUG_MODE

JIRA NVGPUT-76

Change-Id: I054b187f91ee85b09695869e413e43deccd27e5f
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1918080
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svc-misra-checker <svc-misra-checker@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Thomas Fleury <tfleury@nvidia.com>
Tested-by: Thomas Fleury <tfleury@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-16 23:40:40 -07:00
ddutta
80b5e2b8d6 gpu: nvgpu: remove os_fence dependency from channel_sync
Move the wait_cmd_buffer programming for channel_sync->wait_fd to
channel_sync.c.  nvgpu_os_fence->program_waits
interface is now removed. channel_sync can directly retrieve
syncpt/semaphore from the interfaces of struct nvgpu_os_fence_syncpt
and struct nvgpu_os_fence_sema and use it for the wait programming.

Also, change int to u32 for some variables such as num_fences,
max_wait_size and wait_cmd_size.

Jira NVGPU-1093

Change-Id: I19c1b10d676caff49ce57861091f7f0ea65e7676
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1829719
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-16 15:34:13 -07:00
Deepak
7e8ca5f5e7 gpu: nvgpu: Remove cyclic dependency PMU<->GR.
-Created & used HAL for dumping gr falcon stats.
-Trimmed the fecs_dump_falcon_stats to re-use code from
 generic falcon debug dump.

JIRA NVGPU-621

Change-Id: Ia008726915112b33f0aca68a48cb98b8ed2c3475
Signed-off-by: Deepak <dgoyal@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1923353
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: svc-misra-checker <svc-misra-checker@nvidia.com>
Reviewed-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-16 05:54:55 -07:00
Deepak Nibade
0f68b79add gpu: nvgpu: gv100: update gating reglist
Current gating reglist includes both unicast and broadcast registers
Having unicast registers is unnecessary since corresponding broadcast
registers are already being written

Also some of the unicast registers were manually commented out in
auto generated file to avoid PRI timeouts

Re-generate gating reglist with new script which skips unicast
registers so that we don't have to manually update auto generated
file

Bug 2150883

Change-Id: I0b099d23049d8c7154a9b1fb709dd9e6709cdf38
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1922579
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svc-misra-checker <svc-misra-checker@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-12 17:35:12 +05:30
Nitin Kumbhar
dcb2a34200 gpu: nvgpu: fix circular dep of ce2 and gk20a headers
struct gk20a from gk20a.h needs defination of struct gk20a_ce_app
and ce2_gk20a.h needs defination of struct gk20a. This creates
a circular dependency.

Fix this by making gk20a_ce_app a pointer to skip knowing the
complete type details and using forward declarations for struct
gk20a_ce_app and struct gk20a.

The gk20a_ce_app pointer is alloc'ed in gk20a_init_ce_support()
and free'ed in gk20a_ce_destroy.

JIRA NVGPU-611

Change-Id: I4d62d5f2b2d1492db73bae69f90a1fe5586fba76
Signed-off-by: Nitin Kumbhar <nkumbhar@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1917945
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-12 17:35:11 +05:30
Amurthyreddy
c114b9e77e gpu: nvgpu: MISRA 14.4 err/ret/status as boolean
MISRA rule 14.4 doesn't allow the usage of integer types as booleans
in the controlling expression of an if statement or an iteration
statement

Fix violations where the integer variables err, ret, status are used
as booleans in the controlling expression of if and loop statements.

JIRA NVGPU-1019

Change-Id: Ia950828797b8eff4bc754269ea2d9fa272f59436
Signed-off-by: Amurthyreddy <amurthyreddy@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1919111
Reviewed-by: svc-misra-checker <svc-misra-checker@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Scott Long <scottl@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-12 17:35:11 +05:30
Deepak Nibade
92c1949392 gpu: nvgpu: add separate unit for cyclestats_snapshot
Add new separate unit common/perf/cyclestats_snapshot.c and add
corresponding header file include/nvgpu/cyclestats_snapshot.h

This unit is h/w independent and simply calls gops.perf.* HALs
exposed by perf unit to do the h/w configurations

Also remove gv11b/css_gr_gv11b.* files as h/w specific sequence
implemented in them is already moved to perf unit

Rename all cyclestats_snapshot HALs in the form nvgpu_css_*()

Jira NVGPU-1103

Change-Id: I303f6becb313ac918e06c495a5fe299947a1f0b1
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1916652
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-12 17:35:11 +05:30
Alex Waterman
5c52444a42 gpu: nvgpu: Track num_user_mapped_buffers more clearly
This patch moves the increment and decrement of the user mapped
buffer count to the insert/remove mapped buffer functions since
this value should only ever change when these functions are called.

Bug 200105199

Change-Id: I5b0a86d00e9e948c48e313153a668eb2e10fca49
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1917791
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-12 17:35:11 +05:30
aalex
e1a4bc8401 Revert "Revert "gpu: nvgpu: refactor SET_SM_EXCEPTION_MASK ioctl""
This patch was reverted as the "set_sm_exception_type_mask" HAL
assignment for gp10b was missing causing regression on Pascal platform.

Added missing gp10b HAL assignment for setting SM exception mask.

Bug 200447406
This reverts commit ce5228e094.

Change-Id: Ic48f4661fd4b6100310f8b4d23d902847e31f5df
Signed-off-by: aalex <aalex@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1837653
Reviewed-by: svc-misra-checker <svc-misra-checker@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-by: Aparna Das <aparnad@nvidia.com>
Tested-by: Sandarbh Jain <sanjain@nvidia.com>
Reviewed-by: Nirav Patel <nipatel@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-12 17:35:10 +05:30
Nicolas Benech
ea0a0c8485 gpu: nvgpu: Fix all MISRA 17.7 violations in mm
MISRA Rule-17.7 requires the return value of all functions to be used.
Fix is either to use the return value or change the function to return
void. This patch contains fix for all 17.7 violations in MM code.

JIRA NVGPU-677

Change-Id: Iafcbe99050ba79e571107ccf9d2a89bbeb6239b1
Signed-off-by: Nicolas Benech <nbenech@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1847890
Reviewed-by: svc-misra-checker <svc-misra-checker@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-12 17:35:09 +05:30
Terje Bergstrom
3bda3a0678 Revert "Revert "gpu: nvgpu: add turing support""
This reverts commit 278842d6ff4e15467e0b8761c6e1b2a05f926f91.

Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Change-Id: I37f47c137c048ddc3a728e143b6f30be525de120
Reviewed-on: https://git-master.nvidia.com/r/1918622
2018-10-12 17:35:09 +05:30
David Gilhooley
b74a4dbd26 Revert "gpu: nvgpu: add turing support"
This reverts commit 27686d8b56316c7ad772dd91548e91516d59f3b1.

Change-Id: Iebda705858edbd58c10ca3024a4ad060401485b6
Signed-off-by: David Gilhooley <dgilhooley@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1918612
2018-10-12 17:35:09 +05:30
Deepak Nibade
51244d6112 gpu: nvgpu: add turing support
Add Turing specific common, unit, hardware header files

Make all the Makefile and Makefile.sources changes to compile
all Turing specific code

Bug 200454999

Change-Id: I62ebff5c078b4b8817fc83ea0e4ee3cfffe668dc
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1917983
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
2018-10-12 17:35:09 +05:30
Deepak Nibade
362f9a7371 gpu: nvgpu: use MC hals to get MC reset mask in perf unit
In common/perf/perf_*.c we right now include MC h/w headers to get
MC reset mask for perfmon

Use MC HAL gops.mc.reset_mask() to get reset mask instead of using
direct MC register headers

Jira NVGPU-1102

Change-Id: I06f0b3638775c07c9e4e7ac80efd479871322d50
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1852620
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svc-misra-checker <svc-misra-checker@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-12 17:35:08 +05:30
Mahantesh Kumbar
bbf70e1ce9 gpu: nvgpu: Bootstrap SEC2 RTOS & LS falcons
-Call secured_sec2_start() to start SEC2 RTOS ucode execution
 on SEC2 falcon in nvgpu_init_sec2_support() function
-Modified nvgpu_init_pmu_support() to do PMU bootstrap
 from SEC2 RTOS by sending command.
-Added function nvgpu_sec2_bootstrap_ls_falcons() to
 bootstrap LS falcon by taking falcon id as a parameter &
 sending request to SEC2 RTOS with command
 NV_SEC2_ACR_CMD_ID_BOOTSTRAP_FALCON.
-Modified method gr_gm20b_load_ctxsw_ucode() to
 bootstrap FECS & GPCCS falcons using SEC2 RTOS
 in cold boot & recovery path.
-Updated ldr_cfg parameters for SEC2 falcon
-Skip adding PMU ucode details to non-wpr blob preparation
 to skip supporting of LS PMU falcon bootstrap.

JIRA NVGPUT-85

Change-Id: I5f6828e2737e247767814014801671327bb34a4e
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1832363
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-12 17:35:08 +05:30
Mahantesh Kumbar
07cb84214b gpu: nvgpu: SEC2 IPC support
-Created sec2_ipc.c to support SEC2 IPC.
-Defined nvgpu_sec2_cmd_post() to send command
 to SEC2 RTOS from nvgpu along with dependent
 methods like seq acquire/release, validate &
 write cmd.
-Defined nvgpu_sec2_process_message() to
 process message from SEC2 RTOS & route
 to correct handler based on flag.
-Method sec2_process_init_msg() helps fetch
 parameters sent from SEC2 RTOS to setup
 queue, debug buffer as parameters.
-Created sec2 ops under gops to access
 sec2 engine specific HALs.
-Defined nvgpu_sec2_queue_init() init
 command & message for SEC2 RTOS using
 common falcon queue.
-Made Makefile changes to include sec2_ipc.c for build

JIRA NVGPUT-82

Change-Id: I6e4c2d6ec71aa61a543f34680d1412167c9a8cc6
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1828034
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
2018-10-12 17:35:08 +05:30