ANDROID: userfaultfd: abort uffdio ops if mmap_lock is contended
Check if the mmap_lock is contended when looping over the pages that
are requested to be filled. When it is observed, we rely on the already
existing mechanism to return bytes copied/filled and -EAGAIN as error.
This helps by avoiding contention of mmap_lock for long running
userfaultfd operations. The userspace can perform other tasks before
retrying the operation for the remaining pages.
Bug: 320478828
Bug: 331535903
Signed-off-by: Lokesh Gidra <lokeshgidra@google.com>
(cherry picked from https://android-review.googlesource.com/q/commit:6bc28fdfeec3373198d11fae1c9663a598ddb05c)
Merged-In: I6d485fd03c96a826956ee3962e58058be3cf81c1
Change-Id: I6d485fd03c96a826956ee3962e58058be3cf81c1
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 7e7fee4..e8e8a79 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -664,6 +664,15 @@
if (unlikely(err == -ENOENT)) {
void *page_kaddr;
+ /*
+ * Return early due to mmap_lock contention only after
+ * some pages are copied to ensure that jank sensitive
+ * threads don't keep retrying for progress-critical
+ * pages.
+ */
+ if (copied && mmap_lock_is_contended(dst_mm))
+ break;
+
mmap_read_unlock(dst_mm);
BUG_ON(!page);
@@ -688,6 +697,9 @@
if (fatal_signal_pending(current))
err = -EINTR;
+
+ if (mmap_lock_is_contended(dst_mm))
+ err = -EAGAIN;
}
if (err)
break;
