gpu: nvgpu: add nvgpu_start_gpu_idle in nvgpu_remove path

- With ELPG + RG enabled, gpu_module_reload test fails.
- This happens because the test tries to unload nvgpu.ko
  module and then reload it. This all happens with RG enabled.
- During rmmod of nvgpu.ko module the code path taken is:
  nvgpu_remove() ->  nvgpu_quiesce() -> gk20a_pm_prepare_poweroff
  -> nvgpu_prepare_poweroff -> pmu_destroy
- In this code path, NVGPU_DRIVER_IS_DYING flag is not set.
- Thus, in pmu_pg_task thread (which keeps on running in parallel),
  commands are sent to the PMU and the driver keeps waiting for the
  ACK in nvgpu_pmu_wait_fw_ack_status().
- Add nvgpu_start_gpu_idle() in nvgpu_remove() path, before calling
  nvgpu_quiesce().
- This will set NVGPU_DRIVER_IS_DYING flag to true.
- nvgpu_can_busy() will return 0 when the driver is shutting down or
  getting removed.

Bug 3676200

Change-Id: Ic24f58c210e4b477e5d560b053b70c16308e16f1
Signed-off-by: Divya <dsinghatwari@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2762310
Reviewed-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-by: Tejal Kudav <tkudav@nvidia.com>
Reviewed-by: Mahantesh Kumbar <mkumbar@nvidia.com>
Reviewed-by: Vijayakumar Subbu <vsubbu@nvidia.com>
GVS: Gerrit_Virtual_Submit <buildbot_gerritrpt@nvidia.com>
(cherry picked from commit 8f1792565e71b822a6e9cc50af4b43c1b48518e0)
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2765300
Tested-by: Mahantesh Kumbar <mkumbar@nvidia.com>

drivers/gpu/nvgpu/os/linux/module.c

@@ -1943,6 +1943,7 @@ int nvgpu_remove(struct device *dev)
 
 	nvgpu_log_fn(g, " ");
 
+	nvgpu_start_gpu_idle(g);
 	err = nvgpu_quiesce(g);
 	WARN(err, "gpu failed to idle during driver removal");