)]}' { "commit": "b44e46bb047d136bc8977497b6fc2a9f08740321", "tree": "005da71967c83060b64da1908d7a1f3ad72675aa", "parents": [ "54e7412d4ff9af8fc9d2b5c792e0ff6e39dcc045" ], "author": { "name": "Peter Collingbourne", "email": "pcc@google.com", "time": "Tue Jun 08 12:57:27 2021 +1000" }, "committer": { "name": "Hridya Valsaraju", "email": "hridya@google.com", "time": "Tue Jun 15 19:33:15 2021 +0000" }, "message": "FROMGIT: mm: improve mprotect(R|W) efficiency on pages referenced once\n\nIn the Scudo memory allocator [1] we would like to be able to detect\nuse-after-free vulnerabilities involving large allocations by issuing\nmprotect(PROT_NONE) on the memory region used for the allocation when it\nis deallocated. Later on, after the memory region has been \"quarantined\"\nfor a sufficient period of time we would like to be able to use it for\nanother allocation by issuing mprotect(PROT_READ|PROT_WRITE).\n\nBefore this patch, after removing the write protection, any writes to the\nmemory region would result in page faults and entering the copy-on-write\ncode path, even in the usual case where the pages are only referenced by a\nsingle PTE, harming performance unnecessarily. Make it so that any pages\nin anonymous mappings that are only referenced by a single PTE are\nimmediately made writable during the mprotect so that we can avoid the\npage faults.\n\nThis program shows the critical syscall sequence that we intend to use in\nthe allocator:\n\n #include \u003cstring.h\u003e\n #include \u003csys/mman.h\u003e\n\n enum { kSize \u003d 131072 };\n\n int main(int argc, char **argv) {\n char *addr \u003d (char *)mmap(0, kSize, PROT_READ | PROT_WRITE,\n MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);\n for (int i \u003d 0; i !\u003d 100000; ++i) {\n memset(addr, i, kSize);\n mprotect((void *)addr, kSize, PROT_NONE);\n mprotect((void *)addr, kSize, PROT_READ | PROT_WRITE);\n }\n }\n\nThe effect of this patch on the above program was measured on a\nDragonBoard 845c by taking the median real time execution time of 10 runs.\n\nBefore: 2.94s\nAfter: 0.66s\n\nThe effect was also measured using one of the microbenchmarks that we\nnormally use to benchmark the allocator [2], after modifying it to make\nthe appropriate mprotect calls [3]. With an allocation size of 131072\nbytes to trigger the allocator\u0027s \"large allocation\" code path the\nper-iteration time was measured as follows:\n\nBefore: 27450ns\nAfter: 6010ns\n\nThis patch means that we do more work during the mprotect call itself in\nexchange for less work when the pages are accessed. In the worst case,\nthe pages are not accessed at all. The effect of this patch in such cases\nwas measured using the following program:\n\n #include \u003cstring.h\u003e\n #include \u003csys/mman.h\u003e\n\n enum { kSize \u003d 131072 };\n\n int main(int argc, char **argv) {\n char *addr \u003d (char *)mmap(0, kSize, PROT_READ | PROT_WRITE,\n MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);\n memset(addr, 1, kSize);\n for (int i \u003d 0; i !\u003d 100000; ++i) {\n #ifdef PAGE_FAULT\n memset(addr + (i * 4096) % kSize, i, 4096);\n #endif\n mprotect((void *)addr, kSize, PROT_NONE);\n mprotect((void *)addr, kSize, PROT_READ | PROT_WRITE);\n }\n }\n\nWith PAGE_FAULT undefined (0 pages touched after removing write\nprotection) the median real time execution time of 100 runs was measured\nas follows:\n\nBefore: 0.330260s\nAfter: 0.338836s\n\nWith PAGE_FAULT defined (1 page touched) the measurements were\nas follows:\n\nBefore: 0.438048s\nAfter: 0.355661s\n\nSo it seems that even with a single page fault the new approach is faster.\n\nI saw similar results if I adjusted the programs to use a larger mapping\nsize. With kSize \u003d 1048576 I get these numbers with PAGE_FAULT undefined:\n\nBefore: 1.428988s\nAfter: 1.512016s\n\ni.e. around 5.5%.\n\nAnd these with PAGE_FAULT defined:\n\nBefore: 1.518559s\nAfter: 1.524417s\n\ni.e. about the same.\n\nWhat I think we may conclude from these results is that for smaller\nmappings the advantage of the previous approach, although measurable, is\nwiped out by a single page fault. I think we may expect that there should\nbe at least one access resulting in a page fault (under the previous\napproach) after making the pages writable, since the program presumably\nmade the pages writable for a reason.\n\nFor larger mappings we may guesstimate that the new approach wins if the\ndensity of future page faults is \u003e 0.4%. But for the mappings that are\nlarge enough for density to matter (not just the absolute number of page\nfaults) it doesn\u0027t seem like the increase in mprotect latency would be\nvery large relative to the total mprotect execution time.\n\nLink: https://lkml.kernel.org/r/20210527190453.1259020-1-pcc@google.com\nLink: https://linux-review.googlesource.com/id/I98d75ef90e20330c578871c87494d64b1df3f1b8\nLink: [1] https://source.android.com/devices/tech/debug/scudo\nLink: [2] https://cs.android.com/android/platform/superproject/+/master:bionic/benchmarks/stdlib_benchmark.cpp;l\u003d53;drc\u003de8693e78711e8f45ccd2b610e4dbe0b94d551cc9\nLink: [3] https://github.com/pcc/llvm-project/commit/scudo-mprotect-secondary2\nSigned-off-by: Peter Collingbourne \u003cpcc@google.com\u003e\nReviewed-by: Peter Xu \u003cpeterx@redhat.com\u003e\nCc: Kostya Kortchinsky \u003ckostyak@google.com\u003e\nCc: Evgenii Stepanov \u003ceugenis@google.com\u003e\nCc: Andrea Arcangeli \u003caarcange@redhat.com\u003e\nSigned-off-by: Andrew Morton \u003cakpm@linux-foundation.org\u003e\nSigned-off-by: Stephen Rothwell \u003csfr@canb.auug.org.au\u003e\n(cherry picked from commit e2037f9c0c61ed6964bb1291292ae88f073a100c\n https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git akpm)\n[pcc: squashed v4-\u003ev5 diff which appeared as a separate commit: ec7563ea9f6a470e9bb532b024ce29d9474daf24]\nChange-Id: Ic3994b2ec914d3f62f95c1ef338986e350e69e36\nBug: 191165850\n", "tree_diff": [ { "type": "modify", "old_id": "d95115a5728373d8b5009ce5cd38723d0d8c41c5", "old_mode": 33188, "old_path": "mm/mprotect.c", "new_id": "05073d7abff48c2504c4621a4f128d49d6385b46", "new_mode": 33188, "new_path": "mm/mprotect.c" } ] }