Files
linux-nvgpu/drivers/gpu/nvgpu/hal/rc
Alex Waterman 7e99a68e34 gpu: nvgpu: Add basic recovery debugging messages
Add basic recovery messages that describe what's happening during
the recovery process. Hide this under a new recovery specific GPU
debug log flag. The logs look like:

[  276.000733] nvgpu: 17000000.gv11b                gv11b_fifo_recover:162  [DBG]  REC | Recovery starting
[  276.000737] nvgpu: 17000000.gv11b                gv11b_fifo_recover:163  [DBG]  REC |   ID      = 0
[  276.000741] nvgpu: 17000000.gv11b                gv11b_fifo_recover:164  [DBG]  REC |   id_type = TSG
[  276.000745] nvgpu: 17000000.gv11b                gv11b_fifo_recover:165  [DBG]  REC |   rc_type = MMU fault
[  276.000748] nvgpu: 17000000.gv11b                gv11b_fifo_recover:166  [DBG]  REC |   Engine bitmask: 0x0
[  276.000753] nvgpu: 17000000.gv11b                gv11b_fifo_recover:170  [DBG]  REC | Acquiring engines_reset_mutex
[  276.000756] nvgpu: 17000000.gv11b                gv11b_fifo_recover:174  [DBG]  REC | Acquiring runlist_lock for active runlists
[  276.000764] nvgpu: 17000000.gv11b                gv11b_fifo_recover:185  [DBG]  REC | Channels bound to this TSG:
[  276.000767] nvgpu: 17000000.gv11b                gv11b_fifo_recover:190  [DBG]  REC |   0 | chid 511
[  276.001098] nvgpu: 17000000.gv11b                gv11b_fifo_recover:222  [DBG]  REC | PBDMA   Bitmask: 0x1
[  276.001102] nvgpu: 17000000.gv11b                gv11b_fifo_recover:228  [DBG]  REC | Runlist Bitmask: 0x1
[  276.001106] nvgpu: 17000000.gv11b                gv11b_fifo_recover:240  [DBG]  REC | Disabling RL scheduler now
[  276.001126] nvgpu: 17000000.gv11b                gv11b_fifo_recover:246  [DBG]  REC | Disabling CG/PG now
[  276.189348] nvgpu: 17000000.gv11b                gv11b_fifo_recover:259  [DBG]  REC | Clearing PBDMA_FAULTED, ENG_FAULTED in CCSR register
[  276.191972] nvgpu: 17000000.gv11b                gv11b_fifo_recover:264  [DBG]  REC | Disabling TSG
[  276.191983] nvgpu: 17000000.gv11b                gv11b_fifo_recover:279  [DBG]  REC | Preempting runlists for RC
[  276.192001] nvgpu: 17000000.gv11b                gv11b_fifo_recover:288  [DBG]  REC | Polling for TSG to be off PBDMA
[  276.192012] nvgpu: 17000000.gv11b                gv11b_fifo_recover:296  [DBG]  REC |   Done!
[  276.192016] nvgpu: 17000000.gv11b                gv11b_fifo_recover:306  [DBG]  REC | Resetting relevant engines
[  276.192020] nvgpu: 17000000.gv11b                gv11b_fifo_recover:318  [DBG]  REC |   Engine bitmask for RL 0: 0xd
[  276.192024] nvgpu: 17000000.gv11b                gv11b_fifo_recover:323  [DBG]  REC |   > Restting engine: ID=0
[  276.209567] nvgpu: 17000000.gv11b                gv11b_fifo_recover:347  [DBG]  REC |     Done!
[  276.209572] nvgpu: 17000000.gv11b                gv11b_fifo_recover:323  [DBG]  REC |   > Restting engine: ID=2
[  276.214290] nvgpu: 17000000.gv11b                gv11b_fifo_recover:347  [DBG]  REC |     Done!
[  276.214295] nvgpu: 17000000.gv11b                gv11b_fifo_recover:323  [DBG]  REC |   > Restting engine: ID=3
[  276.224986] nvgpu: 17000000.gv11b                gv11b_fifo_recover:347  [DBG]  REC |     Done!
[  276.225013] nvgpu: 17000000.gv11b                gv11b_fifo_recover:377  [DBG]  REC | Re-enabling runlists
[  276.225034] nvgpu: 17000000.gv11b                gv11b_fifo_recover:383  [DBG]  REC | Re-enabling CG/PG
[  276.225134] nvgpu: 17000000.gv11b                gv11b_fifo_recover:394  [DBG]  REC | Releasing engines reset mutex

Note the "REC |" which lets one easily do:

  $ dmesg | grep "REC |"

To get a clear ubobstrructed view of the recovery progress in the dmesg
log.

JIRA NVGPU-5606

Change-Id: I183f2b5ac54edc60ee894a82111723e27aa5c46b
Signed-off-by: Alex Waterman <alexw@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2392991
Reviewed-by: automaticguardword <automaticguardword@nvidia.com>
Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com>
Reviewed-by: svc-mobile-cert <svc-mobile-cert@nvidia.com>
Reviewed-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-by: Tejal Kudav <tkudav@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
GVS: Gerrit_Virtual_Submit
2020-12-15 14:13:28 -06:00
..