| # Arm Memory Tagging Extension (MTE) implementation | |
| AOSP supports Arm MTE to detect invalid memory accesses. The implementation is | |
| spread across multiple components, both within and out of the AOSP tree. This | |
| document gives an overview and pointers about how the various MTE features are | |
| implemented. | |
| For documentation of the behavior rather than the implementation, see the | |
| [SAC page on MTE] instead. For MTE for apps, see the [NDK page on MTE]. | |
| The relevant components are: | |
| * [LLVM Project] (out of AOSP tree) | |
| * Stack tagging instrumentation pass | |
| * Scudo memory allocator | |
| * bionic | |
| * libc | |
| * dynamic loader | |
| * Zygote | |
| * debuggerd | |
| * [NDK] | |
| ## MTE enablement | |
| The way MTE is requested and enabled differs between native binaries and Java | |
| apps. This is necessarily so, because Java apps get forked from the Zygote, | |
| while native executables get inintialized by the linker. | |
| ### Native binaries | |
| Both AOSP and the NDK allow you to compile C/C++ code that use MTE to detect | |
| memory safety issues. The [NDK legacy cmake toolchain] and the | |
| [NDK new cmake toolchain] both support "memtag" as an argument for | |
| `ANDROID_SANITIZE`. NDK make has no specific support for MTE, but the | |
| relevant flags can be passed directly as `CFLAGS` and `LDFLAGS`. | |
| For the OS itself, [Soong] supports "memtag_[heap|stack|globals]" as | |
| `SANITIZE_TARGET and as `sanitize:` attribute in Android.bp files; | |
| [Android make] supports the same environment variables as Soong. This passes | |
| the appropriate flags to the clang driver for both compile and link steps. | |
| #### Linker | |
| * For **dynamic executables** LLD has support to | |
| [add appropriate dynamic sections] as defined in the [ELF standard] | |
| * For **static executables** and as a fallback for older devices, LLD | |
| also supports [adding the Android-specific ELF note] | |
| Both of the above are controlled by the linker flag `--android-memtag-mode` | |
| which is [passed in by the clang driver] if | |
| `-fsanitize=memtag-[stack|heap|globals]` is [passed in]. | |
| `-fsanitize=memtag` [enables all three] (even for API levels that don't | |
| implement the runtime for globals, which means builds from old versions | |
| of clang may no work with newer platform versions that support globals). | |
| `-fsanitize-memtag-mode` allows to choose between ASYNC and SYNC. | |
| This information can be queried using `llvm-readelf --memtag`. | |
| This information is [picked up by libc init] to decide whether to enable MTE. | |
| `-fsanitize-heap` controls both whether scudo tags allocations, and whether | |
| tag checking is enabled. | |
| #### Runtime environment (dynamic loader, libc) | |
| There are two different initialization sequences for libc, both of which end up | |
| calling `__libc_init_mte`. | |
| N.B. the linker has its own copy of libc, which is used when executing these | |
| functions. That is why we have to use `__libc_shared_globals` to communicate | |
| with the libc of the process we are starting. | |
| * **static executables** `__libc_init` is called from `crtbegin.c`, which calls | |
| `__libc_init_mte` | |
| * **dynamic executables** the linker calls `__libc_init_mte` | |
| `__libc_init_mte` figures out the appropriate MTE level that is requested by | |
| the process, calls `prctl` to request this from the kernel, and stores data in | |
| `__libc_shared_globals` which gets picked up later to enable MTE in scudo. | |
| It also does work related to stack tagging and permissive mode, which will be | |
| detailed later. | |
| ### Apps | |
| Apps can request MTE be enabled for their process via the manifest attribute | |
| `android:memtagMode`. This gets interpreted by Zygote, which always runs with | |
| `ASYNC` MTE enabled, because MTE for a process can only be disabled after | |
| it has been initialized (see [Native binaries](#native-binaries)), not enabled. | |
| [decideTaggingLevel] in the Zygote figures out whether to enable MTE for | |
| an app, and stores it in the `runtimeFlags`, which get picked up by | |
| [SpecializeCommon] after forking from the Zygote. | |
| ## MTE implementation | |
| ### Heap Tagging | |
| Heap tagging is implemented in the scudo allocator. On `malloc` and `free`, | |
| scudo will update the memory's tags to prevent use-after-free and buffer | |
| overflows. | |
| [scudo's memtag.h] contains helper functions to deal with MTE tag management, | |
| which are used in [combined.h] and [secondary.h]. | |
| ### Stack Tagging | |
| Stack tagging requires instrumenting function bodies. It is implemented as | |
| an instrumentation pass in LLVM called [AArch64StackTagging], which sets | |
| the tags according to the lifetime of stack objects. | |
| The instrumentation pass also supports recording stack history, consisting of: | |
| * PC | |
| * Frame pointer | |
| * Base tag | |
| This can be used to reconstruct which stack object was referred to in an | |
| invalid access. The logic to reconstruct this can be found in the | |
| [stack script]. | |
| Stack tagging is enabled in one of two circumstances: | |
| * at process startup, if the main binary or any of its dependencies are | |
| compiled with `memtag-stack` | |
| * library compiled with `memtag-stack` is `dlopen`ed later, either directly or | |
| as a dependency of a `dlopen`ed library. In this case, the | |
| [__pthread_internal_remap_stack_with_mte] function is used (called from | |
| `memtag_stack_dlopen_callback`). Because `dlopen` | |
| is handled by the linker, we have to [store a function pointer] to the | |
| process's version of the function in `__libc_shared_globals`. | |
| Enabling stack MTE consists of two operations: | |
| * Remapping the stacks as `PROT_MTE` | |
| * Allocating a stack history buffer. | |
| The first operation is only necessary when the process is running with MTE | |
| enabled. The second operation is also necessary when the process is not running | |
| with MTE enabled, because the writes to the stack history buffer are | |
| unconditional. | |
| libc keeps track of this through two globals: | |
| * `__libc_memtag_stack`: whether stack MTE is enabled on the process, i.e. | |
| whether the stack pages are mapped with PROT\_MTE. This is always false if | |
| MTE is disabled for the process (i.e. `libc_globals.memtag` is false). | |
| * `__libc_memtag_stack_abi`: whether the process contains any code that was | |
| compiled with memtag-stack. This is true even if the process does not have | |
| MTE enabled. | |
| ### Globals Tagging | |
| TODO(fmayer): write once submitted | |
| ### Crash reporting | |
| For MTE crashes, debuggerd serializes special information into the Tombstone | |
| proto: | |
| * Tags around fault address | |
| * Scudo allocation history | |
| This is done in [tombstone\_proto.cpp]. The information is converted to a text | |
| proto in [tombstone\_proto\_to\_text.cpp]. | |
| ## Bootloader control | |
| The bootloader API allows userspace to enable MTE on devices that do not ship | |
| with MTE enabled by default. | |
| See [SAC MTE bootloader support] for the API definition. In AOSP, this API is | |
| implemented in [system/extras/mtectrl]. mtectrl.rc handles the property | |
| changes and invokes mtectrl to update the misc partition to communicate | |
| with the bootloader. | |
| There is also an [API in Device Policy Manager] that allows the device admin | |
| to enable or disable MTE under certain circumstances. | |
| The device can opt in or out of these APIs by a set of system properties: | |
| * `ro.arm64.memtag.bootctl_supported`: the system property API is supported, | |
| and an option is displayed in Developer Options. | |
| * `ro.arm64.memtag.bootctl_settings_toggle`: an option is displayed in the | |
| normal settings. This requires `ro.arm64.memtag.bootctl_supported` to be | |
| true. This implies `ro.arm64.memtag.bootctl_device_policy_manager`, if it | |
| is not explicitely set. | |
| * `ro.arm64.memtag.bootctl_device_policy_manager`: the Device Policy Manager | |
| API is supported. | |
| ## Permissive MTE | |
| Permissive MTE refers to a mode which, instead of crashing the process on an | |
| MTE fault, records a tombstone but then continues execution of the process. | |
| An important caveat is that system calls with invalid pointers (where the | |
| pointer tag does not match the memory tag) still return an error code. | |
| This mode is only available for system services, not apps. It is implemented | |
| in the [debugger\_signal\_handler] by disabling MTE for the faulting thread. | |
| Optionally, the user can ask for MTE to be re-enabled after some time. | |
| This is achieved by arming a timer that calls [enable_mte_signal_handler] | |
| upon expiry. | |
| ## MTE Mode Upgrade | |
| When a system service [crashes in ASYNC mode], we set an impossible signal | |
| as an exit code (because that signal is always gracefully handled by libc), | |
| and [in init] we set `BIONIC_MEMTAG_UPGRADE_SECS`, which gets handled by | |
| [libc startup]. | |
| [SpecializeCommon]: https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/core/jni/com_android_internal_os_Zygote.cpp?q=f:frameworks%2Fbase%2Fcore%2Fjni%2Fcom_android_internal_os_Zygote.cpp%20%22%20mallopt(M_BIONIC_SET_HEAP_TAGGING_LEVEL,%22&ss=android%2Fplatform%2Fsuperproject%2Fmain | |
| [LLVM Project]: https://github.com/llvm/llvm-project/ | |
| [NDK]: https://android.googlesource.com/platform/ndk/ | |
| [NDK legacy cmake toolchain]: https://android.googlesource.com/platform/ndk/+/refs/heads/main/build/cmake/android-legacy.toolchain.cmake#490 | |
| [NDK new cmake toolchain]: https://android.googlesource.com/platform/ndk/+/refs/heads/main/build/cmake/flags.cmake#56 | |
| [Soong]: https://cs.android.com/android/platform/superproject/main/+/main:build/soong/cc/sanitize.go?q=sanitize.go&ss=android%2Fplatform%2Fsuperproject%2Fmain | |
| [decideTaggingLevel]: https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/core/java/com/android/internal/os/Zygote.java?q=symbol:decideTaggingLevel | |
| [picked up by libc init]: https://cs.android.com/android/platform/superproject/main/+/main:bionic/libc/bionic/libc_init_mte.cpp?q=symbol:__get_tagging_level%20f:bionic | |
| [enables all three]: https://github.com/llvm/llvm-project/blob/e732d1ce86783b1d7fe30645fcb30434109505b9/clang/include/clang/Basic/Sanitizers.def#L62 | |
| [passed in]: https://github.com/llvm/llvm-project/blob/ff2e619dfcd77328812a42d2ba2b11c3ff96f410/clang/lib/Driver/SanitizerArgs.cpp#L719 | |
| [passed in by the clang driver]: https://github.com/llvm/llvm-project/blob/ff2e619dfcd77328812a42d2ba2b11c3ff96f410/clang/lib/Driver/ToolChains/CommonArgs.cpp#L1595 | |
| [adding the Android-specific ELF note]: https://github.com/llvm/llvm-project/blob/435cb0dc5eca08cdd8d9ed0d887fa1693cc2bf33/lld/ELF/Driver.cpp#L1258 | |
| [ELF standard]: https://github.com/ARM-software/abi-aa/blob/main/memtagabielf64/memtagabielf64.rst#6dynamic-section | |
| [add appropriate dynamic sections]: https://github.com/llvm/llvm-project/blob/7022498ac2f236e411e8a0f9a48669e754000a4b/lld/ELF/SyntheticSections.cpp#L1473 | |
| [storeTags]: https://cs.android.com/android/platform/superproject/main/+/main:external/scudo/standalone/memtag.h?q=f:scudo%20f:memtag.h%20function:storeTags | |
| [SAC page on MTE]: https://source.android.com/docs/security/test/memory-safety/arm-mte | |
| [NDK page on MTE]: https://developer.android.com/ndk/guides/arm-mte | |
| [AArch64StackTagging]: https://github.com/llvm/llvm-project/blob/main/llvm/lib/Target/AArch64/AArch64StackTagging.cpp | |
| [scudo's memtag.h]: https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/scudo/standalone/memtag.h | |
| [combined.h]: https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/scudo/standalone/combined.h | |
| [secondary.h]: https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/scudo/standalone/secondary.h | |
| [__pthread_internal_remap_stack_with_mte]: https://cs.android.com/android/platform/superproject/main/+/main:bionic/libc/bionic/pthread_internal.cpp?q=__pthread_internal_remap_stack_with_mte | |
| [stack script]: https://cs.android.com/android/platform/superproject/main/+/main:development/scripts/stack?q=stack | |
| [Android make]: https://cs.android.com/android/platform/superproject/main/+/main:build/make/core/config_sanitizers.mk | |
| [store a function pointer]: https://cs.android.com/android/platform/superproject/main/+/main:bionic/libc/bionic/libc_init_dynamic.cpp;l=168?q=memtag_stack_dlopen_callback | |
| [tombstone\_proto.cpp]: https://cs.android.com/android/platform/superproject/main/+/main:system/core/debuggerd/libdebuggerd/tombstone_proto.cpp?q=tombstone_proto.cpp | |
| [tombstone\_proto\_to\_text.cpp]: https://cs.android.com/android/platform/superproject/main/+/main:system/core/debuggerd/libdebuggerd/tombstone_proto_to_text.cpp | |
| [SAC MTE bootloader support]: https://source.android.com/docs/security/test/memory-safety/bootloader-support | |
| [system/extras/mtectrl]: https://cs.android.com/android/platform/superproject/main/+/main:system/extras/mtectrl/ | |
| [API in Device Policy Manager]: https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/core/java/android/app/admin/DevicePolicyManager.java?q=symbol:setMtePolicy%20f:DevicePolicyManager.java | |
| [debuggerd\_signal_handler]: https://cs.android.com/android/platform/superproject/main/+/main:system/core/debuggerd/handler/debuggerd_handler.cpp?q=f:debuggerd_handler.cpp%20symbol:debuggerd_signal_handler | |
| [enable_mte_signal_handler]: https://cs.android.com/android/platform/superproject/main/+/main:bionic/libc/bionic/libc_init_mte.cpp?q=symbol:__enable_mte_signal_handler | |
| [in init]: https://cs.android.com/android/platform/superproject/main/+/main:system/core/init/service.cpp?q=f:system%2Fcore%2Finit%2Fservice.cpp%20should_upgrade_mte | |
| [crashes in ASYNC mode]: https://cs.android.com/android/platform/superproject/main/+/main:system/core/debuggerd/handler/debuggerd_handler.cpp;l=799?q=BIONIC_SIGNAL_ART_PROFILER | |
| [libc startup]: https://cs.android.com/android/platform/superproject/main/+/main:bionic/libc/bionic/libc_init_mte.cpp?q=BIONIC_MEMTAG_UPGRADE_SECS |