The following changes implements the initial (as per bringup) nvlink driver.
(1) SW initialization of nvlink core driver structures
(2) Nvlink interrupt handling
(3) Device initialization (IOCTRL, pll and clocks, device level intr)
(4) Falcon support for minion
(5) Minion load and bootstrapping
(6) Link initialization and DL PROD settings
(7) Device Interface init (and switching HSHUB to nvlink)
(8) HS set/get mode for both link and sublink
(9) Topology discovery and VBIOS settings.
(10) Ensures we get physical contiguous memory when Nvlink is enabled
This driver includes a hack for the current single dev/single link limitation.
JIRA: EVLR-2331
JIRA: EVLR-2330
JIRA: EVLR-2329
JIRA: EVLR-2328
Change-Id: Idca9a819179376cc655784482b24b575a52fa9e5
Signed-off-by: Thomas Fleury <tfleury@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1656790
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Until issue related to low frequencies root caused,
limit min frequency to known safe value: 216.75Mhz.
This change needs to be reverted, once orginal issue
root-caused and fixed.
Bug 2051863
Bug 2056266
Change-Id: If6e56f59ee5fa06967fde1128b58a7fc97be74e9
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1657595
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
On GV11B, CBC base is calculated in similar fashion than it's
calculated on dGPUs. Thus, remove gv11b_ltc_cbc_fix_config() as it
would incorrectly multiply the CBC base by the LTC count.
Bug 2054860
Change-Id: Iaed717161547468c17e12236149d970c497885b3
Signed-off-by: Sami Kiminki <skiminki@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1654506
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Add User space API NVGPU_AS_IOCTL_GET_SYNC_RO_MAP to get read-only syncpoint
address map in user space
We already map whole syncpoint shim to each address space with base address
being vm->syncpt_ro_map_gpu_va
This new API exposes this base GPU_VA address of syncpoint map, and unit size
of each syncpoint to user space.
User space can then calculate address of each syncpoint as
syncpoint_address = base_gpu_va + (syncpoint_id * syncpoint_unit_size)
Note that this syncpoint address is read_only, and should be only used for
inserting semaphore acquires.
Adding semaphore release with this address would result in MMU_FAULT
Define new HAL g->ops.fifo.get_sync_ro_map and set this for all GPUs supported
on Xavier SoC
Bug 200327559
Change-Id: Ica0db48fc28fdd0ff2a5eb09574dac843dc5e4fd
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1649365
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Add characteristic flag NVGPU_GPU_FLAGS_SUPPORT_SYNCPOINT_ADDRESS to indicate if
platform supports semaphore GPU_VA address for a syncpoint
Define NVGPU_SUPPORT_SYNCPOINT_ADDRESS for core driver book keeping
Set this flag for both GV100 and GV11B since Xavier SoC supports a semaphore
GPU_VA address for a syncpoint through syncpoint SHIM
Bug 200327559
Change-Id: I1f31673c9fd59f493d0b35a80d23151fc063ae06
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1649364
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
The GPU has multiple different operating modes in respect to IOMMU'ability.
As such there needs to be a clean way to tell the driver whether it is
IOMMU'able or not. This state also does not always reflect what is possible:
all becasue the GPU can generate IOMMU'ed memory requests doesn't mean it
wants to.
The nvgpu_iommuable() API has now existed for a little while which is a
useful way to convey whether nvgpu should consider the GPU as IOMMU'able.
However, there is also the g->mm.bypass_smmu flag which used to be able to
override what the GPU decided it should do. Typically it was assigned
the same value as nvgpu_iommuable() but that was not necessarily a
requirment.
This patch removes all the usages of g->mm.bypass_smmu and instead uses the
nvgpu_iommuable() function. All places where the check against
g->mm.bypass_smmu have been replaced with nvgpu_iommuable(). The code
should now be much cleaner.
Subsequently other checks can also be placed in the nvgpu_iommuable()
function. For example, when NVLINK comes online and the GPU should no
longer consider DMA addresses and instead use scatter-gather lists
directly the ngpu_iommuable() function will be able to check the state of
NVLINK and then act accordingly.
Change-Id: I0da6262386de15709decac89d63d3eecfec20cd7
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1648332
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
nvlink core library no longer exposes the set_init_state()
interface as it wishes to block init_state changes from
endpoint drivers.
Now, the core driver is responsible for initializing init_state
variables using set_init_state() interface. Hence, we remove
this redundant code.
Change-Id: I81c4922cf48f7918e69795579b39b7fa0c299644
Signed-off-by: Tejal Kudav <tkudav@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1646437
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Adeel Raza <araza@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Set the DMA mask to 34 bits so that large DMA allocs can be done.
Currently the DMA mask is left unset which limits the size of the
maximum DMA allocation to 32 bits.
The 34 bit mask was chosen because it works for all chips (even
gm20b supports 34 bit physical addresses). However, newer chips
could use larger masks in the future if they desire.
Bug 200377221
Change-Id: Iaa0543f77ff4e2bd6616f38e4464240375bb37b6
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1641762
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
After moving devfreq enable to end of finalize power on,
intermittent issues related to gpu booting with devfreq
enabled are fixed.
Enabled devfreq for gv11b by enabling ""nvhost_podgov"
governor in platform data.
Reused scaling functions from gp10b/gk20a.
Removed emc floor on railgate for power saving.
Added max emc frequency as floor in rail-ungate for
faster gpu boot.
Bug 2049965
Bug 2039013
Bug 200377508
Change-Id: Ia1dec278b663b9f7ed859dd953a60f3eae7ef9a0
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1644702
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
VM and CDE code assumes that dma_buf_attachment is stored as a pointer
in the private dma_buf_drvdata, so it is not tracked. In Linux trees
without dma_buf_*_drvdata() support this is not true, so change the
code to explicitly track dma_buf_attachment.
JIRA NVGPU-4
Change-Id: I692f05a19a6469195d5444a7e5ff6e92f77ae272
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1648004
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Enabling gpu scaling driver after finalize poweron,
will make gpu booting happen at initially set
frequency(1GHz).
Also doing platform specific init scale after enabling
scaling driver.
Bug 2049965
Bug 2039013
Bug 200377508
Change-Id: I633f8f5a25d9de18cbb3a022913b8b725ccd87e5
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1644703
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
The message to tell RM server to unbind channel has to be sent after
client unbinds the channel and before client calls tsg release. The
channel has to belong to a tsg on RM server before client submit a
runlist to remove the channel. Or there's a bare channel problem.
By moving .tsg_unbind_channl one layer lower, gk20a_tsg_unbind_channel()
will be common functions for all chip, and it'll call tsg release after
call .tsg_unbind_channel. So vgpu won't need to worry about tsg was
released before sending msg to RM server.
Bug 200382695
Bug 200382785
Change-Id: I32acc122f3f9d5d0628049ccf673225f9e90c87a
Signed-off-by: Richard Zhao <rizhao@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1645383
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Patch 7240b3c2 enabled secure allocation for gv11b
But since we allocate secure buffers in poweron path, and secure allocation
needs GPU to be in off state, this results in deadlock in poweron path
To solve this, we already cause early VPR resize for older chips by calling
gk20a_tegra_secure_page_alloc() from late_probe
Implement same for gv11b.
Add late_probe callback and add a call to gk20a_tegra_secure_page_alloc()
Bug 2038249
Change-Id: I8c17b069962b26edbd0639a7c0d6c2fdaa352935
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1648831
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Tested-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Seema Khowala <seemaj@nvidia.com>
Simplify the copyengine code massively by storing the job post fence
pointers in an array of fences instead of mixing them up in the command
buffer memory. The post fences are used when the ring buffer of a
context gets full and we need to wait for the oldest slot to free up.
NVGPU-43
NVGPU-52
Change-Id: I36969e19676bec0f38de9a6357767a8d5cbcd329
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1646037
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Adds the skeleton and integration of the GV100 endpoint driver to NVGPU
(1) Adds a OS abstraction layer for the internal nvlink structure.
(2) Adds linux specific integration with Nvlink core driver.
(3) Adds function pointers for nvlink api, initialization and isr process.
(4) Adds initial support for minion.
(5) Adds new GPU enable properties to handle NVLINK presence
(6) Adds new GPU enable properties for SG_PHY bypass (required for NVLINK over
PCI)
(7) Adds parsing of nvlink vbios structures.
(8) Adds logging defines for NVGPU
JIRA: EVLR-2328
Change-Id: I0720a165a15c7187892c8c1a0662ec598354ac06
Signed-off-by: David Nieto <dmartineznie@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1644708
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Data can be speculatively loaded from memory and stay in cache even
when bound check fails. This can lead to unintended information
disclosure via side-channel analysis.
To mitigate this problem insert a speculation barrier.
bug 2039126
CVE-2017-5753
Change-Id: I982225e754cc5d430c19f4cc542302e52243bd38
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1640501
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
In the nvgpu_big_free() function the passed in address is checked
to see what type of address it is: kmalloc or vmalloc. This change
uses the is_vmalloc_addr() instead since this is a much clearer and
easier way to determine if a virtual address should be vfree()ed.
Anything not a vmalloc address is then assumed to be a kmalloc()
address.
Bug 2049449
Change-Id: I2bd9441d3c5fc455f03ec2075d012c607280ad5f
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1644802
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Arun Kannan <akannan@nvidia.com>
Reviewed-by: Seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Lots of code paths were split to T19x specific code paths and structs
due to split repository. Now that repositories are merged, fold all of
them back to main code paths and structs and remove the T19x specific
Kconfig flag.
Change-Id: Id0d17a5f0610fc0b49f51ab6664e716dc8b222b6
Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1640606
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Added sw method support for SET_BES_CROP_DEBUG4.
In this sw method:
CLAMP_FP_BLEND_TO_MAXVAL forces overflow and
CLAMP_FP_BLEND_TO_INF blend results to clamp to FP maxval.
Added support for this sw method in gp10b/gp106/gv11b
and gv100.
Bug 2046636
Change-Id: I3a9e97587aca76718f7f504ea3b853f87409092a
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1641529
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Fix a race condition where we'd still be booting up the gpu and/or
initializing the driver but elsewhere assume that all is done already.
Some userspace APIs to make sure that we're ready by testing
g->gr.sw_ready, but this flag is set in the middle of bootup; there are
other things after gr initialization. Add a new flag that is enabled
after bootup is fully complete at the end of finalize_poweron, and
change the checks in user API paths to test the new flag only.
These checks are only in the ioctl paths for ctrl, dbg and tsg, and in
the ctrl device's opening path.
The gr.sw_ready flag is still left there to signify whether just gr has
had its bookkeeping initialized.
Bug 200370011
Change-Id: I2995500e06de46430d9b835de1e9d60b3f01744e
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1640124
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Syncpoints can longer be disabled/enabled during run time as
NVGPU_HAS_SYNCPOINTS flag is set based on has_syncpoints
value in platform data during probe. Based on this, either
of syncpoint or semaphore pool is initialized.
Bug 2040115
Change-Id: Ib256e1a6ec8b1584799adb6f183fd567aebfaf13
Signed-off-by: Seema Khowala <seemaj@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1640380
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Enable devfreq for gv11b by enabling ""nvhost_podgov"
governor in platform data.
Reuse scaling functions from gp10b/gk20a.
Remove emc floor on railgate for power saving and make
max emc frequency as floor in rail-ungate for faster gpu boot.
Bug 2039013
Bug 200377508
Change-Id: I65ee7735202e3decbe3451157f7fc1f1f273c3ff
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1639752
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
