get_cycles is a linux specific API used in common code. This API
is being used, it seems, as a method to generate time stamps. So
add an API to generate 'high resolution' time stamps. This API
returns an opaque time stamp: that is not something one may use
directly as a time since in the Linux implementation we just use
this cycle counter.
Other implementations will, of course, be free to implement as a
real time stamp.
JIRA NVGPU-525
Change-Id: I237aac9bd6c795d000459025bdb4fce92e8aaa3d
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1673811
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
The pre-fence wait for semaphores in the submit path has supported a
fast path for fences that have only one underlying semaphore. The fast
path just inserts the wait on this sema to the pushbuffer directly. For
other fences, the path has been using a CPU wait indirection, signaling
another semaphore when we get the CPU-side callback.
Instead of only supporting prefences with one sema, unroll all the
individual semaphores and insert waits for each to a pushbuffer, like
we've already been doing with syncpoints. Now all sema-backed syncs get
the fast path. This simplifies the logic and makes it more explicit that
only foreign fences need the CPU wait.
There is no need to hold references to the sync fence or the semas
inside: this submitted job only needs the global read-only sema mapping
that is guaranteed to stay alive while the VM of this channel stays
alive, and the job does not outlive this channel.
Jira NVGPU-43
Jira NVGPU-66
Jira NVGPU-513
Change-Id: I7cfbb510001d998a864aed8d6afd1582b9adb80d
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1636345
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
VGPU has set aggressive_sync_destroy_thresh even for GV11B, and that breaks
allocation of user managed syncpoint on VGPU
Remove this check for now until some solution is finalized
Bug 200397265
Bug 200326065
Change-Id: Idd765cfdd40b9055d9e083d59c85c84d8b213ee9
Signed-off-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1675678
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Mikko Perttunen <mperttunen@nvidia.com>
- Added support for single rail multi step volt policy & below
are the list of define & struct added/updated to support same.
CTRL_VOLT_POLICY_TYPE_SINGLE_RAIL_MULTI_STEP 0x04,
NV_VBIOS_VOLTAGE_POLICY_1X_ENTRY_TYPE_SINGLE_RAIL_MULTI_STEP 0x04,
Updated struct vbios_voltage_policy_table_1x_entry,
struct nv_pmu_volt_volt_policy_sr_multi_step_boardobj_set, this holds
members which help to config single rail multi step like delay
between switch step, ramp up & ramp down step size in uv.
- Added case to support SINGLE_RAIL_MULTI_STEP in
volt_volt_policy_construct() based on boardobj type.
- Added case to support SINGLE_RAIL_MULTI_STEP in
volt_get_volt_policy_table() to read data from VBIOS
table vbios_voltage_policy_table_1x_entry & extract to
voltage_policy_single_rail_multi_step.
- Added methods to forward single rail multi step data to
PMU using below methods by assigning data read from
VBIOS voltage_policy_single_rail_multi_step to
nv_pmu_volt_volt_policy_sr_multi_step_boardobj_set
interface.
volt_construct_volt_policy_single_rail_multi_step()
volt_policy_pmu_data_init_sr_multi_step()
volt_policy_pmu_data_init_single_rail()
construct_volt_policy_single_rail()
Change-Id: I17bc8c320777191611365ee63274c38ffe5ecbf7
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1660687
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
- Created volt ops under pmu_ver to support volt_set_voltage,
volt_get_voltage & volt_send_load_cmd_to_pmu.
- Renamed volt load, set_voltage & get_voltage gp10x method names.
- Added new volt load, set_voltage & get_voltage methods for gv10x
using RPC & added code to handle ack in pmu_rpc_handler() along
with struct rail_list changes.
- Updated volt ops of gp106 & gv100 to point to respective methods.
- Added member volt_dev_idx_ipc_vmin & volt_scale_exp_pwr_equ_idx to
"struct nv_pmu_volt_volt_rail_boardobj_set" & "struct voltage_rail"
made changes to update members as needed.
- Added member volt_scale_exp_pwr_equ_idx to
"struct vbios_voltage_rail_table_1x_entry" to read
value from VBIOS table & update rail boardobj set interface.
- Defines for volt RPC "NV_PMU_RPC_ID_VOLT_*"
- Define struct's volt load, set_voltage & get_voltage to execute
volt RPC.
Change-Id: I4a41adcf7536468beaa8a73f551b1d608aabd161
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1659728
Reviewed-by: Automatic_Commit_Validation_User
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
When nvgpu maps an nvgpu_mem struct the nvgpu driver has a choice of
either using a fixed or non-fixed mapping. For non-fixed mappings the
GMMU APIs allocate a VA space for the caller. In that case the GMMU
APIs must also free that VA range when nvgpu unmaps the nvgpu_mem.
For fixed mappings the GMMU APIs must instead not manage the life
time of the VA space. To support these two possibilities add a field
to nvgpu_mem that specifies whether the GMMU APIs must or must not
free the GPU VA range during the GMMU unmap operation.
Also fix a case in the nvgpu vm_area code that would double free a
VA allocation in some cases (sparse allocs).
Change-Id: Idc32dbb8208fa7c1c05823e67b54707fea51c6b7
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1669920
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
- Updated & added new parameter "bool is_copy_back" to
nvgpu_pmu_rpc_execute() to support copy back processed
RPC request from PMU to caller by passing parameter value
true & this blocks method till it receives ACK from PMU
for requested RPC.
- Added "struct rpc_handler_payload" to hold info
required for RPC handler like RPC buff address &
clear memory if copy back is not requested.
- Added define PMU_RPC_EXECUTE_CPB to support to copy back
processed RPC request from PMU to caller.
- Updated RPC callback handler support, crated memory &
assigned default handler if callback is not requested
else use callback parameters data to request to PMU.
- Added define PMU_RPC_EXECUTE_CB to support callback
- Updated pmu_wait_message_cond(), restricted condition
check to 8-bit instead 32-bit condition check.
Change-Id: Ic05289b074954979fd0102daf5ab806bf1f07b62
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1664962
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Modify the GK20A_TRACE_PRINTK config such that it depends on TRACING
instead of FTRACE_PRINTK. The latter is not in upstream Linux nor in our
downstream 4.9, and this option is default n anyway so this is a pretty
safe change.
Change-Id: If4ce5a041c8392d0bc54a60730c6ab3115b0062a
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1674114
GVS: Gerrit_Virtual_Submit
Reviewed-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Add new characteristics flag NVGPU_GPU_FLAGS_SUPPORT_USER_SYNCPOINT to indicate
support for this new API
Add new flag NVGPU_SUPPORT_USER_SYNCPOINT for use of core driver.
Set this flag for VGPU-GV11B
Bug 200326065
Jira NVGPU-179
Change-Id: I6c992b13268b688a2bbc93a3331e987ea2f7dd0c
Signed-off-by: Lakshmanan M <lm@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1670452
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Deepak Nibade <dnibade@nvidia.com>
Tested-by: Jitendra Pratap Singh Chauhan <jchauhan@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
struct nvgpu_semaphore represents (mainly) a threshold value that a sema
at some index will get and struct nvgpu_semaphore_int (aka "hw_sema")
represents the allocation (and write access) of a semaphore index and
the next value that the sema at that index can have. The threshold
object doesn't need a pointer to the sema allocation that is not even
guaranteed to exist for the whole threshold lifetime, so replace the
pointer by the position of the sema in the sema pool.
This requires some modifications to pass a hw sema around explicitly
because it now represents write access more explicitly.
Delete also the index field of semaphore_int because it can be directly
derived from the offset in the sema location and is thus unnecessary.
Jira NVGPU-512
Change-Id: I40be523fd68327e2f9928f10de4f771fe24d49ee
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1658102
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Add IPA to PA translation for GV100 nvlink / pass-through mode
- define platform->phys_addr(g, ipa) method
- call nvgpu_init_soc_vars from nvgpu_tegra_pci_probe
- in nvgpu_init_soc_vars, define set platform->phys_addr to
nvgpu_tegra_hv_ipa_pa, if hypervisor is present.
- in __nvgpu_sgl_phys, use sg_phys, then apply platform->phys_addr
if defined.
- implement IPA to PA translation in nvgpu_tegra_hv_ipa_pa
Bug 200392719
Change-Id: I622049ddc62c2a57a665dd259c1bb4ed3843a537
Signed-off-by: Thomas Fleury <tfleury@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1673582
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Richard Zhao <rizhao@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Add __nvgpu_sgl_phys function that can be used to implement IPA
to PA translation in a subsequent change.
Adapt existing function prototypes to add pointer to gpu context,
as we will need to check if IPA to PA translation is needed.
JIRA EVLR-2442
Bug 200392719
Change-Id: I5a734c958c8277d1bf673c020dafb31263f142d6
Signed-off-by: Thomas Fleury <tfleury@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1673142
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Default semaphore payload size is 16byte. Set it to 4 byte
to avoid double increment of associated sync point with
semaphore release.
Also removed extra 0 op function from syncpoint increment
command.
Bug 2066025
Change-Id: Ia282cc5625827d356b5ba963adb7b1b3c703a931
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1669714
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Replace the padding in nvgpu_channel_wdt_args with a timeout value in
milliseconds, and add NVGPU_IOCTL_CHANNEL_WDT_FLAG_SET_TIMEOUT to
signify the existence of this new field. When the new flag is included
in the value of wdt_status, the field is used to set a per-channel
timeout to override the per-GPU default.
Add NVGPU_IOCTL_CHANNEL_WDT_FLAG_DISABLE_DUMP to disable the long debug
dump when a timed out channel gets recovered by the watchdog. Printing
the dump to serial console takes easily several seconds. (Note that
there is NVGPU_TIMEOUT_FLAG_DISABLE_DUMP about ctxsw timeout separately
for NVGPU_IOCTL_CHANNEL_SET_TIMEOUT_EX as well.)
The behaviour of NVGPU_IOCTL_CHANNEL_WDT is changed so that either
NVGPU_IOCTL_CHANNEL_ENABLE_WDT or NVGPU_IOCTL_CHANNEL_DISABLE_WDT has to
be set. The old behaviour was that other values were silently ignored.
The usage of the global default debugfs-controlled ch_wdt_timeout_ms is
changed so that its value takes effect only for newly opened channels
instead of in realtime. Also, zero value no longer means that the
watchdog is disabled; there is a separate flag for that after all.
gk20a_fifo_recover_tsg used to ignore the value of "verbose" when no
engines were found. Correct this.
Bug 1982826
Bug 1985845
Jira NVGPU-73
Change-Id: Iea6213a646a66cb7c631ed7d7c91d8c2ba8a92a4
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1510898
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
With proper wrap-handling comparisons now supported, it's safe to not
reset a kernel-managed semaphore to 0 when initializing it to be used by
some channel; the value can be left unchanged, so that any pending waits
on other channels for this sema can't get corrupted anymore. This makes
semaphore values very similar to syncpoints, i.e., just monotonically
increasing counters.
Also clear the semaphore sea to values of 0xfffffff0 when allocating it.
This way it takes 16 increments on each sema to wrap over the 32-bit
integer range; such wrapping would eventually happen if the memory was
initialized to zeros, so this way any bugs possibly caused by wrapping
not taken into account would uncover quickly after boot.
Jira NVGPU-514
Change-Id: I93f9b1d32d020a4c23824f5856bc463b1895b99d
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1652087
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: Alex Waterman <alexw@nvidia.com>
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
The value of NV_PERF_PMASYS_MEM_BUMP is different for Volta
and NVGPU_IOCTL_CHANNEL_CYCLE_STATS_SNAPSHOT_CMD_FLUSH did not
have correct behavior on GV11B due to that.
The patch adds an instance of css_hw_set_handled_snapshots
for Volta to fix that.
Bug 1960846
Bug 2068936
Change-Id: Ic057338d3b1b951a66d070267e69a90f136598b9
Signed-off-by: Martin Radev <mradev@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1668568
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Compare gpu semaphores in the kernel in the same way as the hardware
does: released if value is over threshold, but at most half of u32's
range. This makes it possible to skip zeroing the sema values when semas
are allocated, so that they'd be just monotonically increasing numbers
like syncpoints are.
Jira NVGPU-514
Change-Id: I3bae352fbacfe9690666765b9ecdeae6f0813ea1
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1652086
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Alex Waterman <alexw@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
- Created ops for below boardobj methods to support gp10x & gv10x
branch boardobj changes, and defined methods for gv10x with
postfix _v1 with below names
boardobjgrp_pmucmd_construct_impl
boardobjgrp_pmuset_impl
boardobjgrp_pmugetstatus_impl
is_boardobjgrp_pmucmd_id_valid
- These ops are assigned based on PMU version to respective
chip.
- Modified BOARDOBJGRP_PMU_CMD_GRP_SET_CONSTRUCT &
BOARDOBJGRP_PMU_CMD_GRP_GET_STATUS_CONSTRUCT to support
gp10x & gv10x branch changes
- Updated struct boardobjgrp_pmu_cmd to include members
needed for gv10x boardobj changes
- Created "struct nv_pmu_rpc_struct_board_obj_grp_cmd"
to execute BOARD_OBJ_GRP_CMD using RPC.
- Defined method boardobjgrp_pmucmdsend_rpc() to
send BOARD_OBJ_GRP_CMD to PMU.
Change-Id: If2551bdda80e897e7b21d2966881586f3bbc7a9b
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1656511
GVS: Gerrit_Virtual_Submit
Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
- Added ops "pmu.alloc_super_surface" to create
memory space for pmu super surface
- Defined method nvgpu_pmu_sysmem_surface_alloc()
to allocate pmu super surface memory & assigned
to "pmu.alloc_super_surface" for gv100
- "pmu.alloc_super_surface" set to NULL for gp106
- Memory space of size "struct nv_pmu_super_surface"
is allocated during pmu sw init setup if
"pmu.alloc_super_surface" is not NULL &
free if error occur.
- Added ops "pmu_ver.config_pmu_cmdline_args_super_surface"
to describe PMU super surface details to PMU ucode
as part of pmu command line args command if
"pmu.alloc_super_surface" is not NULL.
- Updated pmu_cmdline_args_v6 to include member
"struct flcn_mem_desc_v0 super_surface"
- Free allocated memory for PMU super surface in
nvgpu_remove_pmu_support() method
- Added "struct nvgpu_mem super_surface_buf" to "nvgpu_pmu" struct
- Created header file "gpmu_super_surf_if.h" to include interface
about pmu super surface, added "struct nv_pmu_super_surface"
to hold super surface members along with rsvd[x] dummy space
to sync members offset with PMU super surface members.
Change-Id: I2b28912bf4d86a8cc72884e3b023f21c73fb3503
Signed-off-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1656571
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
When adding a sema wait to a pushbuf, verify that the sema threshold has
been incremented from the original value by reading the incremented
field instead of value (which is set to nonzero by
nvgpu_semaphore_incr()). Value could be 0 even after an increment if new
semas weren't reset to 0.
Jira NVGPU-514
Change-Id: I295451fbc7eb9e597aea12d73074e99f74a6a899
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1658100
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
/sys/devices/gpu.0/gfxp_wfi_timeout_unit
usec - microseconds
sysclk - gpu clock count
Treat gr_fe_gfxp_wfi_timeout_r as context-switched
register on gv11b.
Set default gfxp_wfi_timeout to 100 usec to match
gp10b at 1GHz.
bug 1888344
Signed-off-by: Kirill Artamonov <kartamonov@nvidia.com>
Change-Id: I7fa64ce6912ae861244856807543b17bd7a26bed
Reviewed-on: https://git-master.nvidia.com/r/1651517
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Check these conditions for fecs arbiter idle:
1. Wait for gr_fecs_arb_ctx_cmd to idle
2. Wait for gr_fecs_ctxsw_status_1:arb_busy to idle
Just waiting for condition 1, only guarantees dispatching
of command but not completion of work.
Bug 2056266
Change-Id: I3fcbdb9cda91fee0ff345a24bf23ba7dabf268d0
Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/1667550
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
