gpu: nvgpu: handle ioctl l2_fb_ops better

Background: There is a race that occurs when l2_fb_ops ioctl is
invoked. The race occurs as part of the flush() call while a
gk20_idle() is in progress.

This patch handles the race by making changes in the l2_fb_ops
ioctl itself. For cases where pm_runtime is disabled or railgate is
disabled, we allow this ioctl call to always go ahead as power is
assumed to be always on.

For the other case, we first check the status of g->power_on. In the
driver, g->power_on is set to true, once unrailgate is completed and is
set to false just before calling railgate.

For linux, the driver invokes gk20a_idle() but there is a delay after
which the call to the rpm_suspend()'s callback gets triggered. This
leads to a scenario where we cannot efficiently rely on the
runtime_pm's APIs to allow us to block an imminent suspend or exit if
the suspend is currently in progress. Previous attempts at solving this
has lead to ineffective solutions and make it much complicated to
maintain the code.

With regards to the above, this patch attempts to simplify the way this
can be solved. The patch calls gk20a_busy() when g->power_on = true.
This prevents the race with gk20a_idle(). Based on the rpm_resume and
rpm_suspend's upstream code, resume is prioritized over a suspend
unless a suspend is already in progress i.e. the delay period has been
served and the suspend invokes the callback. There is a very small
window for this to happen and the ioctl can then power_up the device as
evident from the gk20a_busy's calls.

nvgpu power state is queried using nvgpu_is_powered_off to determine
whether to skip the resume. power state is protected under spinlock.

Bug 200507468

Change-Id: I5c02dfa8ea855732e59b759d167152cf45a1131f
Signed-off-by: Debarshi Dutta <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2299545
(cherry picked from commit 06942bd268)
Signed-off-by: Sagar Kamble <skamble@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2425682
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Reviewed-by: automaticguardword <automaticguardword@nvidia.com>
Reviewed-by: Deepak Nibade <dnibade@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
GVS: Gerrit_Virtual_Submit

drivers/gpu/nvgpu/os/linux/ioctl_ctrl.c

@@ -54,6 +54,7 @@
 #include <nvgpu/soc.h>
 #include <nvgpu/nvgpu_init.h>
 #include <nvgpu/user_fence.h>
+#include <nvgpu/nvgpu_init.h>
 
 #include "ioctl_ctrl.h"
 #include "ioctl_dbg.h"
@@ -700,12 +701,23 @@ static int nvgpu_gpu_ioctl_l2_fb_ops(struct gk20a *g,
 	    (!args->l2_flush && args->l2_invalidate))
 		return -EINVAL;
 
+	/* In case of railgating enabled, exit if nvgpu is powered off */
+	if (nvgpu_is_enabled(g, NVGPU_CAN_RAILGATE) && nvgpu_is_powered_off(g)) {
+		return 0;
+	}
+
+	err = gk20a_busy(g);
+	if (err != 0) {
+		nvgpu_err(g, "failed to take power ref");
+		return err;
+	}
+
 	if (args->l2_flush) {
 		err = g->ops.mm.cache.l2_flush(g, args->l2_invalidate ?
 								true : false);
 		if (err != 0) {
 			nvgpu_err(g, "l2_flush failed");
-			return err;
+			goto out;
 		}
 	}
 
@@ -713,10 +725,13 @@ static int nvgpu_gpu_ioctl_l2_fb_ops(struct gk20a *g,
 		err = g->ops.mm.cache.fb_flush(g);
 		if (err != 0) {
 			nvgpu_err(g, "mm.cache.fb_flush() failed err=%d", err);
-			return err;
+			goto out;
 		}
 	}
 
+out:
+	gk20a_idle(g);
+
 	return err;
 }