ANDROID: mm: Fix VMA ref count after fast-mremap
Since the cmpxchg() to unlock the VMA (reset ref count from -1), is
enclosed in VM_BUG_ON_VMA() it gets compiled out in non-debug builds
(CONFIG_DEBUG_VM=n). This means that any VMA that underwent a fast-remap
will have it's refcount stuck at -1, making it not be eligible for
future speculative faults, and preventing freeing of the VMA.
Bug: 322411509
Bug: 324273560
Change-Id: If5bf61c7d94268700f2c4f096d946201b68abdb8
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
(cherry picked from commit 62865d5e3527d72caf7882a18c18101e7ade406b)
diff --git a/mm/mremap.c b/mm/mremap.c
index 540fb64..5eb3bc2 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -226,7 +226,7 @@
* If we have the only reference, swap the refcount to -1. This
* will prevent other concurrent references by get_vma() for SPFs.
*/
- return atomic_cmpxchg(&vma->file_ref_count, 0, -1) == 0;
+ return atomic_cmpxchg_acquire(&vma->file_ref_count, 0, -1) == 0;
}
/*
@@ -234,12 +234,13 @@
*/
static inline void unlock_vma_ref_count(struct vm_area_struct *vma)
{
+ int old = atomic_xchg_release(&vma->file_ref_count, 0);
+
/*
* This should only be called after a corresponding,
* successful trylock_vma_ref_count().
*/
- VM_BUG_ON_VMA(atomic_cmpxchg(&vma->file_ref_count, -1, 0) != -1,
- vma);
+ VM_BUG_ON_VMA(old != -1, vma);
}
#else /* !CONFIG_SPECULATIVE_PAGE_FAULT */
static inline bool trylock_vma_ref_count(struct vm_area_struct *vma)
