Memory Fragmentation Analyzer Overview

Overview

amemdiff is a host-based tool for analyzing and comparing Android memory usage across different device configurations. Its primary goal is to provide a consistent and automated way to measure the impact of changes like page size (4 KB vs. 16 KB) on memory fragmentation and overall system memory health.

The tool works by connecting to two Android devices via ADB, applying a series of device configurations, running predefined workloads (e.g., opening common apps), and executing system probes (e.g., reading /proc/meminfo) to collect memory data. The collected data is then saved to an output directory for further analysis.

Important Considerations

Local Execution: It is strongly recommended to run amemdiff directly on your local machine and not over an SSH tunnel. The tool needs to interactively prompt for device selection and may have issues with ADB device discovery when tunneled.

Quick Start

Connect two android devices to your machine.
- It's highly recommended to use a physical device.
- When running with a cuttlefish device, specify the following configs to avoid triggering fastboot mode:
```
--device-configs=StayAwakeEnabled,BackgroundProcessLimit1,DisableMendelExperiments,DisableGmsCore,SkipSetupWizard,DisablePersistentLogging,SetBrightnessTo100
```
- You can run an amemdiff on a single device and switch to 4/16 KB between runs but this feature is pending.
Ensure both devices are available over ADB.
- Run adb devices.
- Ensure the target devices are available.
Build amemdiff. From the root of the Android source tree:
```
m amemdiff
```
Fetch the device socket address for each Android device.
- From adb devices, note down the socket address. The format is localhost:12345.
Run amemdiff From the root of the Android source tree:
```
out/host/linux-x86/bin/amemdiff -d=d1name/localhost:12345,d2name/localhost:54321
```
This run uses all default options:
- 4 runs per device per workload.
- Configures the device to avoid unnecessary variance and processing.
- Results will be stored in data_out/. The data_out/ directory will contain the following files after a run:
  - run_config.txt: Details of the amemdiff run configuration.
  - device_4kb_details.txt: Device details for the 4KB device.
  - device_16kb_details.txt: Device details for the 16KB device.
  - device_4kb_SteadyStateWorkload_SystemProbeProcMeminfo.csv: Raw /proc/meminfo data for the 4KB device during the SteadyStateWorkload.
  - device_4kb_SteadyStateWorkload_SystemProbeProcMeminfo_mean.csv: Mean /proc/meminfo data for the 4KB device during the SteadyStateWorkload.
  - device_4kb_SteadyStateWorkload_SystemProbeProcMeminfo_mean_diff.csv: Difference in mean /proc/meminfo data for the 4KB device during the SteadyStateWorkload.
  - device_4kb_SteadyStateWorkload_SystemProbeShowmap.csv: Raw showmap data for the 4KB device during the SteadyStateWorkload.
  - device_4kb_SteadyStateWorkload_SystemProbeShowmap_mean.csv: Mean showmap data for the 4KB device during the SteadyStateWorkload.
  - device_4kb_SteadyStateWorkload_SystemProbeShowmap_mean_diff.csv: Difference in mean showmap data for the 4KB device during the SteadyStateWorkload.
  - device_16kb_SteadyStateWorkload_SystemProbeProcMeminfo.csv: Raw /proc/meminfo data for the 16KB device during the SteadyStateWorkload.
  - device_16kb_SteadyStateWorkload_SystemProbeProcMeminfo_mean.csv: Mean /proc/meminfo data for the 16KB device during the SteadyStateWorkload.
  - device_16kb_SteadyStateWorkload_SystemProbeProcMeminfo_mean_diff.csv: Difference in mean /proc/meminfo data for the 16KB device during the SteadyStateWorkload.
  - device_16kb_SteadyStateWorkload_SystemProbeShowmap.csv: Raw showmap data for the 16KB device during the SteadyStateWorkload.
  - device_16kb_SteadyStateWorkload_SystemProbeShowmap_mean.csv: Mean showmap data for the 16KB device during the SteadyStateWorkload.
  - device_16kb_SteadyStateWorkload_SystemProbeShowmap_mean_diff.csv: Difference in mean showmap data for the 16KB device during the SteadyStateWorkload.

Architectural Overview

sequenceDiagram
    participant User
    participant amemdiff.py
    participant AndroidDevice
    participant RunnablePromptDeviceSocketAddress
    participant RunnableApplyDeviceConfigs
    participant RunnableDismissSetupWizard
    participant RunnableRunWorkload
    participant RunnableRunSystemProbe
    participant RunnableCalculateResultDataAcrossRuns
    participant utils_results.py

    User->>amemdiff.py: Executes amemdiff.py with args
    amemdiff.py->>amemdiff.py: Parses args and gets run_config
    amemdiff.py->>AndroidDevice: Creates AndroidDevice objects
    Note right of amemdiff.py: for device in android_devices
    amemdiff.py->>AndroidDevice: get_results(run_config)
    AndroidDevice->>RunnablePromptDeviceSocketAddress: process()
    AndroidDevice->>RunnableApplyDeviceConfigs: process()
    AndroidDevice->>RunnableDismissSetupWizard: process()
    Note right of AndroidDevice: for workload in run_config.workloads
    Note right of AndroidDevice: for run in run_config.runs_per_workload
    AndroidDevice->>RunnableRunWorkload: process()
    AndroidDevice->>RunnableRunSystemProbe: process()
    AndroidDevice->>RunnableRunWorkload: post_process()
    AndroidDevice->>RunnableCalculateResultDataAcrossRuns: process()
    AndroidDevice-->>amemdiff.py: Returns results
    amemdiff.py->>utils_results.py: write_all_results(results)
    utils_results.py-->>amemdiff.py:
    amemdiff.py-->>User: Prints execution time

Missing logcat logs for kernel boot memory

logcat logs are used to pull kernel boot memory details. This approach is brittle but requires zero set up on the device. If you get a warning that a logcat log stat is missing, you have a few options:

Option 1 - Do Nothing

Kernel boot memory won‘t affect the userspace memory details. If you’re okay without knowing the exact device‘s total RAM size, it’s okay to leave kernel boot memory details missing from the output. It will show ‘None’ in the output.

Option 2 - Capture Early logcat Logs

Kernel memory logs in logcat occur very early during boot of the device. It‘s likely that the ring buffer is truncating the early logcat logs. Below is an example of a logcat log necessary to fetch the kernel’s carveout memory and as you can see, it starts at time 0.

In order to prevent the kernel's ring buffer from truncating the logcat log, you can increase the ring buffer size using the following command. Just know that this will increase the resulting kernel memory footprint.

adb shell setprop persist.logd.size 4M

Run the command below.
Check the timestamp. It should start around 0.
Look for a similar log.

Command:

adb reboot && adb wait-for-device && adb root && adb shell logcat -b kernel -d

Example log:

01-01 11:11:11.000 0 0 I kfence  : initialized - using 524288 bytes for 63 objects...