Merge remote-tracking branch 'joel/master' into mymerge
Change-Id: I06506cff55f261badd394abc2b23a8dc15ce7dce
diff --git a/BCC.md b/BCC.md
new file mode 100644
index 0000000..01b5a22
--- /dev/null
+++ b/BCC.md
@@ -0,0 +1,186 @@
+BCC (BPF compiler collection) for Android
+=========================================
+
+Introduction
+------------
+BCC is a compiler and a toolkit, containing powerful kernel tracing tools that
+trace at the lowest levels, including adding hooks to functions in kernel space
+and user space to deeply understand system behavior while being low in
+overhead. [Here's a presentation with an
+overview](http://www.joelfernandes.org/resources/bcc-ospm.pdf) and visit [BCC's
+project page](https://github.com/iovisor/bcc) for the official BCC
+documentation.
+
+Quick Start
+-----------
+adeb is the primary vehicle for running BCC on Android. It supports
+preparing the target Android device with necessary kernel headers, cloning and
+building BCC on device, and other setup. Take a look a quick look at [adeb
+README](https://github.com/joelagnel/adeb/blob/master/README.md) so that
+you're familiar with what it is.
+
+To download a prebuilt filesystem with BCC already built/installed for an ARM64
+device, you can just run:
+```
+adeb prepare --full
+```
+
+This downloads the FS and also downloads prebuilt kernel headers after
+detecting your device's kernel version. Running BCC this way may cause a warning
+at startup since the headers may not be an *exact* match for your kernel's
+sublevel (only version and patchlevel are matched), however it works well in
+our testing and could be used, as long as you can tolerate the warning.
+
+If you would like to setup your own kernel headers and prevent the warning,
+you can point adeb to the kernel sources which will extract headers from there:
+```
+adeb prepare --full --kernelsrc /path/to/kernel-source/
+```
+For targets other than ARM64, see the [Other Architectures
+section](https://github.com/joelagnel/adeb/blob/master/BCC.md#other-architectures-other-than-arm64)
+
+Now to run BCC, just start an adeb shell: `adeb shell`. This uses adb
+as the backend to start a shell into your adeb environment. Try running
+`opensnoop` or any of the other BCC tracers to confirm that the setup worked
+correctly.
+
+If building your own kernel, following are the kernel requirements:
+
+You need kernel 4.9 or newer. Anything less needs backports. Your kernel needs
+to be built with the following config options at the minimum:
+```
+CONFIG_KPROBES=y
+CONFIG_KPROBE_EVENT=y
+CONFIG_BPF_SYSCALL=y
+```
+Optionally,
+```
+CONFIG_UPROBES=y
+CONFIG_UPROBE_EVENT=y
+```
+Additionally, for the criticalsection BCC tracer to work, you need:
+```
+CONFIG_DEBUG_PREEMPT=y
+CONFIG_PREEMPTIRQ_EVENTS=y
+```
+
+Build BCC during adeb install (Optional)
+--------------------------------------------
+If you would like the latest upstream BCC built and installed on your Android
+device, you can run:
+```
+adeb prepare --build --bcc --kernelsrc /path/to/kernel-source/
+```
+NOTE: This is a slow process and can take a long time. Since it not only builds
+BCC but also installs all non-BCC debian packages onto the filesystem and configures them.
+
+Other Architectures (other than ARM64)
+-----------------------
+By default adeb assumes the target Android device is based on ARM64
+processor architecture. For other architectures, use the --arch option. For
+example for x86_64 architecture, run:
+```
+adeb prepare --arch amd64 --build --bcc --kernelsrc /path/to/kernel-source/
+```
+Note: The --download option ignores the --arch flag. This is because we only
+provide pre-built filesystems for ARM64 at the moment.
+Note: If you pass --arch, you have to pass --build, because prebuilt
+filesystems are not available for non-arm64 devices.
+
+Common Issues
+-------------
+Here are some common issues you may face when running different BCC tools.
+
+* Issue 1: Headers are missing on the target device.
+
+Symptom: This will usually result in an error like the following:
+```
+root@localhost:/# criticalstat
+
+In file included from <built-in>:2
+In file included from /virtual/include/bcc/bpf.h:12:
+In file included from include/linux/types.h:5:
+include/uapi/linux/types.h:4:10: fatal error: 'asm/types.h' file not found
+
+#include <asm/types.h>
+
+ ^~~~~~~~~~~~~
+1 error generated.
+Traceback (most recent call last):
+
+ File "./criticalstat.py", line 138, in <module>
+ b = BPF(text=bpf_text)
+ File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 297, in __init__
+ raise Exception("Failed to compile BPF text:\n%s" % text)
+Exception: Failed to compile BPF text:
+
+#include <uapi/linux/ptrace.h>
+#include <uapi/linux/limits.h>
+#include <linux/sched.h>
+
+extern char _stext[];
+```
+
+* Issue 2: `CONFIG_KPROBES` isn't enabled.
+
+Symptom: This will result in an error like the following:
+```
+Traceback (most recent call last):
+ File "/usr/share/bcc/tools/cachetop", line 263, in <module>
+ curses.wrapper(handle_loop, args)
+ File "/usr/lib/python2.7/curses/wrapper.py", line 43, in wrapper
+ return func(stdscr, *args, **kwds)
+ File "/usr/share/bcc/tools/cachetop", line 172, in handle_loop
+ b.attach_kprobe(event="add_to_page_cache_lru", fn_name="do_count")
+ File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 543, in
+attach_kprobe
+ fn = self.load_func(fn_name, BPF.KPROBE)
+ File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 355, in
+load_func
+ (func_name, errstr))
+Exception: Failed to load BPF program do_count: Invalid argument
+```
+
+* Issue 3: `CONFIG_BPF_SYSCALL` isn't enabled.
+
+Symptom: This may result in a compilation error like the following:
+```
+root@localhost:/# cachetop
+Traceback (most recent call last):
+ File "/usr/share/bcc/tools/cachetop", line 263, in <module>
+ curses.wrapper(handle_loop, args)
+ File "/usr/lib/python2.7/curses/wrapper.py", line 43, in wrapper
+ return func(stdscr, *args, **kwds)
+ File "/usr/share/bcc/tools/cachetop", line 171, in handle_loop
+ b = BPF(text=bpf_text)
+ File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 297, in __init__
+ raise Exception("Failed to compile BPF text:\n%s" % text)
+Exception: Failed to compile BPF text:
+
+
+ #include <uapi/linux/ptrace.h>
+ struct key_t {
+ u64 ip;
+ u32 pid;
+ u32 uid;
+ char comm[16];
+ };
+
+ BPF_HASH(counts, struct key_t);
+
+ int do_count(struct pt_regs *ctx) {
+ struct key_t key = {};
+ u64 zero = 0 , *val;
+ u64 pid = bpf_get_current_pid_tgid();
+ u32 uid = bpf_get_current_uid_gid();
+
+ key.ip = PT_REGS_IP(ctx);
+ key.pid = pid & 0xFFFFFFFF;
+ key.uid = uid & 0xFFFFFFFF;
+ bpf_get_current_comm(&(key.comm), 16);
+
+ val = counts.lookup_or_init(&key, &zero); // update counter
+ (*val)++;
+ return 0;
+ }
+```
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000..d645695
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,202 @@
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
diff --git a/METADATA b/METADATA
new file mode 100644
index 0000000..90881ff
--- /dev/null
+++ b/METADATA
@@ -0,0 +1,19 @@
+name: "adeb"
+description: "adeb (also known as androdeb) provides a powerful Linux shell
+environment where one can run popular and mainstream Linux tracing, compiling,
+editing and other development tools on an existing Android device. All the
+commands typically available on a modern Linux system are supported in adeb."
+
+third_party {
+ url {
+ type: HOMEPAGE
+ value: "https://github.com/joelagnel/adeb/README.md"
+ }
+ url {
+ type: GIT
+ value: "https://github.com/joelagnel/adeb.git"
+ }
+ version: "v0.99f"
+ last_upgrade_date { year: 2018 month: 7 day: 25 }
+ license_type: NOTICE
+}
diff --git a/MODULE_LICENSE_APACHE2 b/MODULE_LICENSE_APACHE2
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/MODULE_LICENSE_APACHE2
diff --git a/NOTICE b/NOTICE
new file mode 100644
index 0000000..d645695
--- /dev/null
+++ b/NOTICE
@@ -0,0 +1,202 @@
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..23ae44d
--- /dev/null
+++ b/README.md
@@ -0,0 +1,173 @@
+adeb
+--------
+
+**adeb** (also known as **androdeb**) provides a powerful Linux shell
+environment where one can run popular and mainstream Linux tracing, compiling,
+editing and other development tools on an existing Android device. All the
+commands typically available on a modern Linux system are supported in
+adeb.
+
+Usecases
+--------
+1. Powerful development environment with all tools ready to go (editors,
+compilers, tracers, perl/python etc) for your on-device development.
+
+2. No more cross-compiler needed: Because it comes with gcc and clang, one can
+build target packages natively without needing to do any cross compilation. We even
+ship git, and have support to run apt-get to get any missing development packages
+from the web.
+
+3. Using these one can run popular tools such as BCC that are difficult to run
+in an Android environment due to lack of packages, dependencies and
+cross-compilation needed for their operation. [Check BCC on Android using
+adeb](https://github.com/joelagnel/adeb/blob/master/BCC.md) for more
+information on that.
+
+4. No more crippled tools: Its often a theme to build a static binary with
+features disabled, because you couldn't cross-compile the feature's dependencies. One
+classic example is perf. However, thanks to adeb, we can build perf natively
+on device without having to cripple it.
+
+Requirements for running
+------------------------
+Target:
+An ARM64 android N or later device which has "adb root" supported. Typically
+this is a build in a userdebug configuration. Device should have atleast 2 GB
+free space in the data partition. If you would like to use other architectures,
+see the [Other Architectures](https://github.com/joelagnel/adeb/blob/master/README.md#how-to-use-adeb-for-other-architectures-other-than-arm64) section.
+
+Host:
+A machine running recent Ubuntu or Debian, with 4GB of memory and 4GB free space.
+Host needs debootstrap and qemu-debootstrap packages.
+To install it, run `sudo apt-get install qemu-user-static debootstrap`.
+Other distributions may work but they are not tested.
+
+Quick Start Instructions
+------------------------
+* First clone this repository into adb and cd into it.
+```
+cd adeb
+
+# Add some short cuts:
+sudo ln -s $(pwd)/adeb /usr/bin/adeb
+
+# Cached image downloads result in a huge speed-up. These are automatic if you
+# cloned the repository using git. However, if you downloaded the repository
+# as a zip file (or you want to host images elsewere), you could set the
+# ADEB_REPO_URL environment variable in your bashrc file.
+# Disclaimer: Google is not liable for the below URL and this
+# is just an example.
+export ADEB_REPO_URL="github.com/joelagnel/adeb/"
+```
+
+* Installing adeb onto your device:
+First make sure device is connected to system
+Then run, for the base image:
+```
+adeb prepare
+```
+The previous command only downloads and installs the base image.
+Instead if you want to download and install the full image, do:
+```
+adeb prepare --full
+```
+
+* Now run adeb shell to enter your new environment!:
+```
+adeb shell
+```
+
+* Once done, hit `CTRL + D` and you will exit out of the shell.
+To remove adeb from the device, run:
+```
+adeb remove
+```
+If you have multiple devices connected, please add `-s <serialnumber>`.
+Serial numbers of all devices connected can be obtained by `adb devices`.
+
+* To update an existing adeb clone on your host, run:
+```
+adeb git-pull
+```
+
+More advanced usage instructions
+--------------------------------
+### Install kernel headers in addition to preparing adeb device:
+```
+adeb prepare --kernelsrc /path/to/kernel-source
+```
+
+### Update kernel headers onto an already prepared device:
+
+If you need to put kernel sources for an existing install, run:
+```
+adeb prepare --kernelsrc /path/to/kernel-source --skip-install
+```
+Note: The kernel sources should have been built (atleast build should have started).
+
+### Build and prepare device with a custom rootfs locally:
+
+The adeb fs will be prepared locally by downloading packages as needed:
+```
+adeb prepare --build
+```
+This is unlike the default behavior, where the adeb rootfs is itself pulled from the web.
+
+If you wish to do a full build (that is locally prepare a rootfs with all packages, including bcc, then do):
+```
+adeb prepare --full --build
+```
+
+### Add kernel headers to device in addition to building locally:
+```
+adeb prepare --build --kernelsrc /path/to/kernel-source/
+```
+
+### Build/install a base image with BCC:
+```
+adeb prepare --build --bcc --kernelsrc /path/to/kernel-source/
+```
+Note: BCC is built from source. Also `--kernelsrc` is recommended for tools to
+function unless device has them already.
+
+### Extract the FS from the device, after its prepared:
+```
+adeb prepare --buildtar /path/
+```
+After device is prepared, it will extract the root fs from it
+and store it as a tar archive at `/path/adeb-fs.tgz`. This
+can be used later.
+
+### Use a previously prepared adeb rootfs tar from local:
+```
+adeb prepare --archive /path/adeb-fs.tgz
+```
+
+### Build a standalone raw EXT4 image out of the FS:
+```
+adeb prepare --build-image /path/to/image.img
+```
+This can then be passed to Qemu as -hda. Note: This option doesn't need a
+device connected.
+
+### How to use adeb for other Architectures (other than ARM64)
+By default adeb assumes the target Android device is based on ARM64
+processor architecture. For other architectures, use the --arch option. For
+example for x86_64 architecture, run:
+```
+adeb prepare --build --arch amd64 --bcc --kernelsrc /path/to/kernel-source/
+```
+Note: The --download option ignores the --arch flag. This is because we only
+provide pre-built filesystems for ARM64 at the moment.
+
+Common Trouble shooting
+-----------------
+1. Installing g++ with `apt-get install g++` fails.
+
+Solution: Run `adeb shell apt-get update` after the `adeb prepare` stage.
+
+2. It's too slow to use debootstrap to create debian fs
+
+Solution: Use a local mirror, for example in China you could use
+https://mirror.tuna.tsinghua.edu.cn/debian/ instead of debian official website
+http://deb.debian.org/debian/
diff --git a/README.version b/README.version
new file mode 100644
index 0000000..3b30f4a
--- /dev/null
+++ b/README.version
@@ -0,0 +1 @@
+URL: https://github.com/joelagnel/adeb/
diff --git a/TODO b/TODO
new file mode 100644
index 0000000..2efc767
--- /dev/null
+++ b/TODO
@@ -0,0 +1,5 @@
+TODO:
+ - update androdeb with latest bcc (which will include criticalstat)
+ - symlink whichever perf is installed to /usr/bin/perf
+ - patch perf with the new futex contention script
+ (probably it can just be dropped into a path)
diff --git a/addons/bashrc b/addons/bashrc
new file mode 100644
index 0000000..5ca6461
--- /dev/null
+++ b/addons/bashrc
@@ -0,0 +1,21 @@
+# The bannered bashrc
+source .bashrc.common
+
+if [ ! -f .banner.shown ]; then
+
+echo ""
+echo "##########################################################"
+echo "# Welcome to androdeb environment running on Android! #"
+echo "# Questions to: Joel Fernandes <joel@joelfernandes.org> #"
+echo " #"
+echo " Try running vim, gcc, clang, git, make, perf, filetop #"
+echo " ..etc or apt-get install something. #"
+echo "##########################################################"
+echo ""
+
+touch .banner.shown
+
+fi
+
+homedir=$( getent passwd "$USER" | cut -d: -f6 )
+export HOME=$homedir
diff --git a/addons/bashrc.common b/addons/bashrc.common
new file mode 100644
index 0000000..14e09d3
--- /dev/null
+++ b/addons/bashrc.common
@@ -0,0 +1,37 @@
+export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/share/bcc/tools/
+export TMPDIR=/tmp/
+
+psk=/proc/sys/kernel/kptr_restrict
+if [ -f $psk ]; then echo 0 > $psk; fi
+
+override_vers=1
+this_vers=$(/get_kvers.sh)
+if [ -d /kernel-headers/ ]; then
+ export BCC_KERNEL_SOURCE=/kernel-headers/;
+
+ h_vers=0
+ vers_line=$(grep LINUX_VERSION_CODE /kernel-headers/include/generated/uapi/linux/version.h)
+ if [ "x$vers_line" != "x" ]; then
+ h_vers=$(echo $vers_line | cut -d ' ' -f3)
+ fi
+
+ if [ $this_vers -eq $h_vers ]; then
+ override_vers=0
+ else
+ this_major=$(($this_vers / (256 * 256)))
+ this_minor=$(($this_vers / 256 % 256))
+ h_major=$(($h_vers / (256 * 256)))
+ h_minor=$(($h_vers / 256 % 256))
+ if [ $this_major -ne $h_major ] || [ $this_minor -ne $h_minor ]; then
+ echo "Error: kernel version does not match installed headers ($this_major.$this_minor <> $h_major.$h_minor)"
+ echo "BCC will not work"
+ override_vers=0
+ fi
+ fi
+fi
+
+# Override kernel version if we haven't found headers with them.
+# Needed for BCC to work on slightly mismatched kernels.
+if [ $override_vers -eq 1 ]; then
+ export BCC_LINUX_VERSION_CODE=$this_vers
+fi
diff --git a/addons/bashrc.silent b/addons/bashrc.silent
new file mode 100644
index 0000000..1e52500
--- /dev/null
+++ b/addons/bashrc.silent
@@ -0,0 +1,4 @@
+source .bashrc.common
+
+homedir=$( getent passwd "$USER" | cut -d: -f6 )
+export HOME=$homedir
diff --git a/addons/build-debian-tar b/addons/build-debian-tar
new file mode 100755
index 0000000..2c58c37
--- /dev/null
+++ b/addons/build-debian-tar
@@ -0,0 +1,19 @@
+#!/system/bin/sh
+set -e
+# Build a tarball out of a android environment which I can then
+# upload to places for people who want to expedite the install
+
+# This script runs on the device
+
+spath=$( cd "$(dirname "$0")" ; pwd -P )
+cd $spath
+
+./device-umount-all
+
+if [ ! -d debian ]; then echo "Error: environment to tar doesn't exist"; exit 1; fi
+
+rm -rf debian-tar; cp -r debian debian-tar;
+rm -rf debian/debian; mv debian-tar debian/debian
+./run-command "tar -zcf androdeb-fs.tgz --exclude='debian/kernel-headers' debian"
+mv debian/androdeb-fs.tgz .
+rm -rf debian/debian-tar
diff --git a/addons/device-umount-all b/addons/device-umount-all
new file mode 100755
index 0000000..fe3b092
--- /dev/null
+++ b/addons/device-umount-all
@@ -0,0 +1,16 @@
+#!/system/bin/sh
+
+if [ "$1x" == "--debugx" ]; then
+ set -x
+fi
+
+umount_all() {
+ mpoints=$(mount|cut -d ' ' -f3|grep debian)
+ for m in $mpoints;
+ do umount $m 2>&1 > /dev/null
+ done
+}
+
+for i in $(seq 0 6); do
+ umount_all 2>&1 > /dev/null
+done
diff --git a/addons/device-unpack b/addons/device-unpack
new file mode 100755
index 0000000..14ba258
--- /dev/null
+++ b/addons/device-unpack
@@ -0,0 +1,28 @@
+#!/system/bin/sh
+
+set -e
+
+# Script to do unpack of rootfs, ensures proper tear down
+# of existing environment. Expects debian rootfs in
+# /data/deb.tar.gz which it will delete after successful
+# unpack of rootfs.
+
+spath=$( cd "$(dirname "$0")" ; pwd -P )
+
+if [ ! -f /data/androdeb/deb.tar.gz ]; then
+ echo "Debian rootfs tar doesn't existing at /data/deb.tar.gz"
+ echo "Run androdeb with device connected first"
+ exit 1
+fi
+
+if [ -d /data/androdeb/debian ]; then
+ echo "androdeb environment already exists, doing a tear down"
+ /data/androdeb/device-umount-all
+ rm -rf /data/androdeb/debian
+fi
+
+
+tar -zxf /data/androdeb/deb.tar.gz -C /data/androdeb/ || die 2 "Couldn't unpack due to tar -x errors"
+rm /data/androdeb/deb.tar.gz
+
+echo "Unpack of rootfs successful!"
diff --git a/addons/get_kvers.sh b/addons/get_kvers.sh
new file mode 100755
index 0000000..3d18e32
--- /dev/null
+++ b/addons/get_kvers.sh
@@ -0,0 +1,12 @@
+#!/bin/bash
+
+kvers=$(uname -r)
+
+MAJOR=$(echo $kvers | awk -F. '{ print $1 }')
+MINOR=$(echo $kvers | awk -F. '{ print $2 }')
+SUBVR=$(echo $kvers | awk -F. '{ print $3 }' | awk -F- '{ print $1 }' | sed 's/[^0-9]*//g')
+
+maj_num=$(($MAJOR * 65536))
+min_num=$(($MINOR * 256))
+
+echo $(($maj_num + $min_num + $SUBVR))
diff --git a/addons/run b/addons/run
new file mode 100755
index 0000000..9738504
--- /dev/null
+++ b/addons/run
@@ -0,0 +1,7 @@
+#!/system/bin/sh
+spath=$( cd "$(dirname "$0")" ; pwd -P )
+cd $spath
+
+source run.common
+
+chroot debian/ /bin/bash
diff --git a/addons/run-command b/addons/run-command
new file mode 100755
index 0000000..6d5a8ca
--- /dev/null
+++ b/addons/run-command
@@ -0,0 +1,10 @@
+#!/system/bin/sh
+spath=$( cd "$(dirname "$0")" ; pwd -P )
+cd $spath
+
+source run.common
+
+# Directly execute a command within the chroot of an Android device
+CMD="$*"
+
+chroot debian /bin/bash --rcfile '.bashrc.silent' -i -c "$CMD"
diff --git a/addons/run.common b/addons/run.common
new file mode 100644
index 0000000..5c45308
--- /dev/null
+++ b/addons/run.common
@@ -0,0 +1,42 @@
+do_mounts()
+{
+ mount --bind /proc debian/proc/ > /dev/null
+
+ mount --bind /dev debian/dev/ > /dev/null
+ mount --bind /dev/pts debian/dev/pts > /dev/null
+
+ mount --bind /sys debian/sys/ > /dev/null
+ mount --bind /sys/fs/bpf/ debian/sys/fs/bpf/ > /dev/null
+ mount --bind /sys/kernel/debug/ debian/sys/kernel/debug/ > /dev/null
+ mount --bind /sys/kernel/debug/tracing/ debian/sys/kernel/debug/tracing/
+
+ # Fix up weirdness with debugfs permission changing because of
+ # above mounts.
+ chmod 0777 /sys/kernel/debug > /dev/null
+ chmod 0777 debian/sys/kernel/debug > /dev/null
+ chmod 0777 /sys/kernel/debug/tracing > /dev/null
+ chmod 0777 debian/sys/kernel/debug/tracing > /dev/null
+
+ # Mount Android partitions
+ if [ -d /d/ ]; then
+ if [ ! -d debian/d ]; then ln -s /sys/kernel/debug debian/d; fi
+ fi
+
+ if [ -d /data/ ]; then
+ mkdir -p debian/data/
+ mount --bind /data debian/data/
+ fi
+
+ if [ -d /system/ ]; then
+ mkdir -p debian/system/
+ mount --bind /system debian/system/
+ fi
+
+ if [ -d /vendor/ ]; then
+ mkdir -p debian/vendor/
+ mount --bind /vendor debian/vendor/
+ fi
+}
+
+mount | grep debian > /dev/null
+if [ $? -ne 0 ]; then do_mounts; fi
diff --git a/adeb b/adeb
new file mode 100755
index 0000000..a651772
--- /dev/null
+++ b/adeb
@@ -0,0 +1,6 @@
+#!/bin/bash -e
+#
+# (c) Joel Fernandes <joel@joelfernandes.org>
+
+spath="$(dirname "$(readlink -f "$0")")"
+source $spath/androdeb
diff --git a/androdeb b/androdeb
new file mode 100755
index 0000000..3600a5b
--- /dev/null
+++ b/androdeb
@@ -0,0 +1,386 @@
+#!/bin/bash -e
+#
+# (c) Joel Fernandes <joel@joelfernandes.org>
+
+VERSION=v0.99g
+
+spath="$(dirname "$(readlink -f "$0")")"
+# spath=$( cd "$(dirname "$0")" ; pwd -P )
+curdir=$( pwd -P )
+source $spath/utils/support
+source $spath/utils/banners
+
+# Set default vars
+DISTRO=buster; ARCH=arm64
+ADB="adb"
+FULL=0 # Default to a minimal install
+DOWNLOAD=1 # Default to downloading from web
+SKIP_DEVICE=0 # Skip device preparation
+INSTALL_BCC=0 # Decide if BCC is to be installed
+
+# Default packages
+PACKAGES=""
+DEFAULT_PACKAGES="bash ca-certificates apt net-tools iputils-ping procps vim"
+
+EXTRA_FILES="none"
+
+config_full_build() {
+ for f in $(ls $spath/packages); do source $spath/packages/$f; done;
+}
+
+# Parse command line parameters
+if [ $# -lt 1 ]; then usage; fi; POSITIONAL=()
+while [[ $# -gt 0 ]]; do key="$1";
+
+# If its shell mode, any future args become shell's args
+if [ "x$ASHELL" == "x1" ]; then
+ if [ -z "$SHELL_ARGS" ]; then
+ SHELL_ARGS=$key
+ else
+ SHELL_ARGS="$SHELL_ARGS $key"
+ fi
+ shift || true; continue
+fi
+
+case $key in
+ shell) ASHELL=1; shift || true; ;;
+ remove) REMOVE=1; shift || true; ;;
+ git-pull) GIT_PULL=1; shift || true; ;;
+ pull) PULL=1; shift || true; break ;;
+ push) PUSH=1; shift || true; break ;;
+ prepare) PREPARE=1; shift || true; ;;
+ --full) FULL=1; config_full_build; shift || true; ;;
+ --arch) ARCH=$2; shift || true; shift || true; ;;
+ --archive) DOWNLOAD=0; TARF=$2; shift || true; shift || true; ;;
+ --bcc) FULL=1; source $spath/packages/bcc; shift || true; ;;
+ --kernelsrc) KERNELSRC="$2"; shift || true; shift || true; ;;
+ --skip-install) SKIP_INSTALL=1; shift || true; ;;
+ --kernel-headers-targz) KERNELHDRS=$2; shift || true; shift || true; ;;
+ --tempdir) TDIR="$2"; shift || true; shift || true; ;;
+ --build) DOWNLOAD=0; shift || true; ;;
+ --buildtar) BTAR=1; DOWNLOAD=0; TARDIR="$2"; shift || true; shift || true; ;;
+ --device|-s) ADB="$ADB -s $2"; shift || true; shift || true; ;;
+ --build-image) BI=1; BUILD_IMAGEF=$2; SKIP_DEVICE=1; DOWNLOAD=0; shift || true; shift || true; ;;
+ --debug) set -x; shift || true; ;;
+ *) c_error "Unknown option ($1)"; usage; ;;
+esac
+done
+
+[ -z $ASHELL ] && box_out "adeb: $VERSION"
+
+if [ $FULL -eq 1 ]; then
+ FNAME=androdeb-fs.tgz.zip
+ FNAME_UZ=androdeb-fs.tgz
+else
+ FNAME=androdeb-fs-minimal.tgz.zip
+ FNAME_UZ=androdeb-fs-minimal.tgz
+fi
+
+if [ ! -z $BTAR ] && [ -z $TARDIR ]; then
+ TARDIR=$spath
+fi
+
+if [ ! -z "$GIT_PULL" ]; then
+ c_info "Updating androdeb by git pull"
+ cd $spath
+ git pull
+ c_info "Done."
+ exit 0
+fi
+
+if [ ! -z "$PULL" ]; then
+ if [ $1 == "-a" ]; then
+ PRESERVE="-a"
+ c_info "Preserving filestamps and mode"
+ shift || true
+ fi
+ file_count=`count_sources $@`
+ i=0
+ while [ $i -lt $file_count ]; do
+ files["$i"]=/data/androdeb/debian/$1
+ shift || true
+ i=$((i + 1))
+ done
+ $ADB pull $PRESERVE "${files[@]}" "$@"
+ exit 0
+fi
+
+if [ ! -z "$PUSH" ]; then
+ file_count=`count_sources $@`
+ i=0
+ while [ $i -lt $file_count ]; do
+ files["$i"]=$1
+ shift || true
+ i=$((i + 1))
+ done
+ dest=/data/androdeb/debian/$1
+ $ADB push $sync "${files[@]}" $dest
+ exit 0
+fi
+
+if [[ ! -z ${TARDIR+x} ]] && [[ ! -d $TARDIR ]]; then die 7 "Tar dir specified doesn't exist"; fi
+
+if [ -z $BI ]; then
+ [ -z $ASHELL ] && c_info "Looking for device.."
+ set +e
+ do_adb_root "$ADB"
+
+ if [ $? -ne 0 ]; then
+ c_error "adb root failed, make sure:"
+ c_error " * If multiple devices connected, provide --device <serialno> (or -s <serialno>)"
+ c_error " * Try to run \"adb root\" manually and see if it works. Typically this needs a userdebug build."
+ c_error ""
+ c_error "Note: adb can be typically obtained using the android-tools-adb or the adb"
+ c_error "packages on your distro, or by installing the Android SDK."
+ die 3 "Exiting."
+ fi
+ set -e
+else
+ [ ! -z $BUILD_IMAGEF ] || die 8 "--build-image passed but no image file provided"
+fi
+
+if [ ! -z "$REMOVE" ]; then
+ die_if_no_androdeb "Nothing to remove."
+ $ADB shell /data/androdeb/device-umount-all || true;
+ $ADB shell rm -rf /data/androdeb; exit 0; fi
+
+##########################################################
+# SHELL
+##########################################################
+if [ ! -z ${ASHELL+x} ]; then
+ set +e; $ADB shell ls /data/androdeb/debian/.bashrc > /dev/null 2>&1
+ if [ $? -ne 0 ]; then
+ die 2 "Device doesn't have an androdeb environment, run \"./androdeb prepare\" first";
+ fi; set -e
+
+ if [ ! -z ${SHELL_ARGS+x} ]; then
+ # Explanation of quotes:
+ # Outer quote is so that androdeb's bash passes the SHELL_ARGS as a single
+ # argument to $ADB shell. Inner quotes is so that run-command can receive all
+ # the args even though they may be separated by spaces. \m/
+ $ADB shell -t /data/androdeb/run-command "\"$SHELL_ARGS\""
+ else
+ $ADB shell -t /data/androdeb/run
+ fi
+
+ exit 0
+fi
+
+##########################################################
+# PREPARE
+##########################################################
+
+function do_cleanup() {
+ rm -rf $TDIR/*; if [ $MKTEMP -eq 1 ]; then rm -rf $TDIR; fi
+}
+
+function push_unpack_headers() {
+ die_if_no_androdeb "Couldn't update headers."
+
+ c_info "Storing kernel headers into androdeb /kernel-headers/"
+ $ADB shell rm -rf /data/androdeb/debian/kernel-headers/
+ $ADB shell mkdir /data/androdeb/debian/kernel-headers/
+ run_quiet $ADB push $TDIR_ABS/kh.tgz /data/androdeb/
+ $ADB shell tar -xvf /data/androdeb/kh.tgz -C /data/androdeb/debian/kernel-headers/ > /dev/null
+ $ADB shell rm /data/androdeb/kh.tgz
+}
+
+function push_unpack_tarred_headers() {
+ die_if_no_androdeb "Couldn't update headers."
+
+ $ADB shell rm -rf /data/androdeb/debian/kernel-headers/
+ $ADB shell mkdir /data/androdeb/debian/kernel-headers/
+
+ c_info "Pushing headers tar onto device"
+ run_quiet $ADB push $1 /data/androdeb/
+
+ c_info "Storing kernel headers into androdeb root directory"
+ $ADB shell tar -xvf /data/androdeb/$(basename $1) -C /data/androdeb/debian/ > /dev/null
+
+ $ADB shell rm /data/androdeb/$(basename $1)
+}
+
+function all_done_banner() {
+ c_info "All done! Run \"adeb shell\" to enter environment"
+}
+
+function detect_repo_url() {
+ ADEB_REPO_URL=`cd $spath && git config -l | grep -m1 remote | grep url | sed -e "s/.*url=//" \
+ -e "s/.*@//" \
+ -e "s/https:\/\///" \
+ -e "s/:/\//" \
+ -e "s/\.git$//"`"/"
+ c_info "Detected URL: $ADEB_REPO_URL"
+}
+
+function check_repo_url () {
+ if [ -z $ADEB_REPO_URL ]; then
+ c_info "No repository URL provided in enviromnent. Attempting to auto-detect it"
+ detect_repo_url
+ fi
+
+ if [ -z $ADEB_REPO_URL ]; then
+ c_warning "Automatic download is disabled. To enable it, please set the \$ADEB_REPO_URL"
+ c_warning "environment variable as recommended in the setup instructions in the README.md"
+ do_cleanup
+ exit 0
+ fi
+}
+
+function download_headers() {
+ KERNEL_MAJOR=`$ADB shell uname -r | cut -d - -f 1 | cut -d . -f 1`
+ KERNEL_MINOR=`$ADB shell uname -r | cut -d - -f 1 | cut -d . -f 2`
+ KERNEL_VERSION="$KERNEL_MAJOR.$KERNEL_MINOR"
+ PREBUILT_HEADERS_FILE=headers-$ARCH-$KERNEL_VERSION.tar.gz.zip
+
+ check_repo_url
+
+ curl -L https://$ADEB_REPO_URL/releases/download/$VERSION/$PREBUILT_HEADERS_FILE --output $TDIR_ABS/$PREBUILT_HEADERS_FILE ||
+ die 9 "Failed to download kernel headers. Please check your internet connection and repository URL"
+
+ unzip -e $TDIR_ABS/$PREBUILT_HEADERS_FILE -d $TDIR_ABS/ ||
+ die 10 "Failed to download kernel headers. Kernel $KERNEL_VERSION for $ARCH may not be supported or ADEB_REPO_URL is incorrect."
+ KERNELHDRS=$TDIR_ABS/`echo "$PREBUILT_HEADERS_FILE" | sed "s/.zip//"`
+}
+
+# Prepare is the last command checked
+if [ -z "$PREPARE" ]; then usage; fi
+
+if [ ! -z "$TARF" ] && [ ! -f $TARF ] && [ -z "$DOWNLOAD" ]; then die 5 "archive provided doesn't exist"; fi
+
+if [ ! -z "$KERNELSRC" ] && [ ! -d $KERNELSRC ]; then die 6 "Kernel source directory provided doesn't exist"; fi
+
+if [ ! -z "$KERNELHDRS" ] && [ ! -f $KERNELHDRS ]; then die 7 "Kernel headers tar.gz doesn't exist"; fi
+
+print_prepare_banner
+
+# Where do we want to store temporary files
+MKTEMP=0; if [[ -z ${TDIR+x} ]] || [[ ! -d "${TDIR}" ]]; then
+ TDIR=`mktemp -d`; MKTEMP=1; fi
+rm -rf $TDIR/*
+TDIR_ABS=$( cd "$TDIR" ; pwd -P )
+
+if [ $DOWNLOAD -eq 1 ]; then
+ c_info "Downloading Androdeb from the web..."; c_info ""
+
+ # Github dropped tar gz support! ##?#??#! Now we've to zip everything.
+ check_repo_url
+
+ curl -L https://$ADEB_REPO_URL/releases/download/$VERSION/$FNAME --output $TDIR_ABS/$FNAME ||
+ die 9 "Failed to download adeb release."
+
+ unzip -e $TDIR_ABS/$FNAME -d $TDIR_ABS/ ||
+ die 10 "Failed to download adeb release. Double check the ADEB_REPO_URL value."
+ TARF=$TDIR_ABS/$FNAME_UZ
+fi
+
+if [ ! -z "$FULL" ] && [ -z "$KERNELSRC" ] && [ -z "$KERNELHDRS" ] && [ -z "$BI" ]; then
+ c_info "Kernel headers are needed but none were provided. Downloading pre-built headers"
+ download_headers
+fi
+
+OUT_TMP=$TDIR/debian; rm -rf $OUT_TMP; mkdir -p $OUT_TMP
+
+# Unpack the supplied kernel headers tar.gz directly into androdeb root
+if [ ! -z "$KERNELHDRS" ]; then
+ c_info "Building updating kernel headers from supplied tar.gz ($KERNELHDRS)"
+
+ # Is header tar gz update the only thing left to do?
+ if [[ ! -z "$SKIP_INSTALL" ]]; then
+ c_info "Skipping install"
+ push_unpack_tarred_headers $KERNELHDRS; do_cleanup; all_done_banner; exit 0; fi
+
+ tar -xvf $KERNELHDRS -C $OUT_TMP/ > /dev/null
+fi
+
+# Package kernel headers
+if [ ! -z "$KERNELSRC" ]; then
+ c_info "Building and updating kernel headers from kernel source dir ($KERNELSRC)"
+ $spath/bcc/build-kheaders-targz.sh ${KERNELSRC} $TDIR_ABS/kh.tgz > /dev/null
+
+ # Is header update the only thing left to do?
+ if [[ ! -z "$SKIP_INSTALL" ]]; then
+ c_info "Skipping install"
+ push_unpack_headers; do_cleanup; all_done_banner; exit 0; fi
+
+ mkdir $OUT_TMP/kernel-headers
+ tar -xvf $TDIR_ABS/kh.tgz -C $OUT_TMP/kernel-headers/ > /dev/null
+fi
+
+# Build FS from existing tar, very simple.
+if [ ! -z "$TARF" ]; then
+ c_info "Using archive at $TARF for filesystem preparation"
+ $ADB shell mkdir -p /data/androdeb/
+
+ c_info "Pushing filesystem to device.."
+ run_quiet $ADB push $TARF /data/androdeb/deb.tar.gz
+
+ c_info "Pushing addons to device.."
+ run_quiet $ADB push $spath/addons/* /data/androdeb/
+
+ c_info "Unpacking filesystem in device.."
+ run_quiet $ADB shell /data/androdeb/device-unpack
+
+ if [ ! -z "$KERNELHDRS" ]; then push_unpack_tarred_headers $KERNELHDRS; fi
+ if [ ! -z "$KERNELSRC" ]; then push_unpack_headers; fi
+
+ do_cleanup; all_done_banner; exit 0
+fi
+
+PACKAGES+="$DEFAULT_PACKAGES"
+c_info "Using temporary directory: $TDIR"
+
+if [[ $EUID -ne 0 ]]; then c_info "The next stage runs as sudo, please enter password if asked."; fi
+
+ex_files=$(mktemp); echo $EXTRA_FILES > $ex_files
+
+sudo $spath/buildstrap $ARCH $DISTRO $TDIR $OUT_TMP \
+ "$(make_csv "$PACKAGES")"\
+ $ex_files $INSTALL_BCC $SKIP_DEVICE
+rm $ex_files
+
+# If we only wanted to prepare a rootfs and don't have
+# a device connected, then just echo that and skip cleanup
+if [ $SKIP_DEVICE -eq 1 ]; then
+ c_info "Device preparation is being skipped for the selected options"
+ c_info "any builds that need to happen on device may be cloned but not built."
+
+ if [ ! -z $BI ]; then
+ sudo $spath/buildimage $OUT_TMP $(dirname $BUILD_IMAGEF)/$(basename $BUILD_IMAGEF)
+ sudo chmod a+rw $(dirname $BUILD_IMAGEF)/$(basename $BUILD_IMAGEF)
+ c_info "Your .img has been built! Enjoy!"
+ fi
+
+ do_cleanup
+ exit 0
+fi
+
+# Push tar to device and start unpack
+$ADB shell mkdir -p /data/androdeb/
+$ADB push $TDIR/deb.tar.gz /data/androdeb/
+$ADB push $spath/addons/* /data/androdeb/
+$ADB shell /data/androdeb/device-unpack
+
+# Build BCC and install bcc on device if needed
+if [ $INSTALL_BCC -eq 1 ]; then
+ $ADB shell /data/androdeb/run-command /bcc-master/build-bcc.sh;
+fi
+
+# Extract a tar of the built, compiled and installed androdeb env
+if [[ ! -z ${TARDIR+x} ]]; then
+ c_info "Creating tarball"
+ pushd $TARDIR
+ if [ $INSTALL_BCC -eq 0 ]; then
+ mv $TDIR/deb.tar.gz $FNAME_UZ
+ else
+ $ADB shell /data/androdeb/build-debian-tar
+ $ADB pull /data/androdeb/androdeb-fs.tgz $FNAME_UZ
+ $ADB shell rm /data/androdeb/androdeb-fs.tgz;
+ fi
+ zip -r $FNAME $FNAME_UZ
+ popd
+fi
+
+do_cleanup
+
+all_done_banner
diff --git a/bcc/build-bcc.sh b/bcc/build-bcc.sh
new file mode 100755
index 0000000..a3ddc83
--- /dev/null
+++ b/bcc/build-bcc.sh
@@ -0,0 +1,12 @@
+#!/bin/bash
+# This script should run within a bcc checkout
+
+spath=$( cd "$(dirname "$0")" ; pwd -P )
+cd $spath
+
+rm -rf build && mkdir -p build && cd build
+cmake .. -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_C_COMPILER=clang-6.0 -DCMAKE_CXX_COMPILER=clang++-6.0
+make -j4
+make install
+cd ..
+rm -rf build
diff --git a/bcc/build-kheaders-targz.sh b/bcc/build-kheaders-targz.sh
new file mode 100755
index 0000000..06eed93
--- /dev/null
+++ b/bcc/build-kheaders-targz.sh
@@ -0,0 +1,39 @@
+#!/bin/bash
+script_full_path=$( cd "$(dirname "$0")" ; pwd -P )
+
+if [ $# -ne 2 ]; then
+ echo "illegal number of parameters, usage: ./build KERNEL_PATH out.tar.gz"
+ exit 1
+fi
+
+# Please provide absolute paths
+KERNEL_PATH=$1
+OUT_TAR=$2
+
+KERNEL_PATH="$(dirname $(readlink -e $KERNEL_PATH))/$(basename $KERNEL_PATH)"
+if [ ! -d "$KERNEL_PATH" ]; then
+ echo "Kernel directory couldn't be found"
+ exit 3
+fi
+
+# kdir=$(basename $KERNEL_PATH)
+
+cd $KERNEL_PATH
+find arch -name include -type d -print | xargs -n1 -i: find : -type f > /tmp/kernel-headers.h
+find include >> /tmp/kernel-headers.h
+
+grep "include/generated/autoconf.h" /tmp/kernel-headers.h > /dev/null 2>&1
+retgrep=$?
+if [ $retgrep -ne 0 ]; then
+ >&2 echo ""
+ >&2 echo "The kernel sources at ${KERNEL_PATH} you pointed to aren't configured and built."
+ >&2 echo "Please atleast run in your kernel sources:"
+ >&2 echo $'make defconfig\nmake'
+ >&2 echo $'\nNote: You dont need to do the full build since headers are generated early on.\n'
+ >&2 echo "Note: Please build your kernel in tree (build and source should be in same directory)"
+ >&2 echo ""
+ exit $retgrep
+fi
+
+cat /tmp/kernel-headers.h | tar -zcf $OUT_TAR -T -
+rm /tmp/kernel-headers.h
diff --git a/bcc/misc/android-futex-contention-record b/bcc/misc/android-futex-contention-record
new file mode 100644
index 0000000..a12ff79
--- /dev/null
+++ b/bcc/misc/android-futex-contention-record
@@ -0,0 +1,2 @@
+#!/bin/bash
+perf record -a -g -e syscalls:sys_enter_futex -e syscalls:sys_exit_futex -e sched:sched_waking $@
diff --git a/bcc/misc/android-futex-contention-report b/bcc/misc/android-futex-contention-report
new file mode 100644
index 0000000..7a8c122
--- /dev/null
+++ b/bcc/misc/android-futex-contention-report
@@ -0,0 +1,4 @@
+#!/bin/bash
+# description: futext contention measurement
+
+perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/android-futex-contention.py
diff --git a/bcc/misc/android-futex-contention.py b/bcc/misc/android-futex-contention.py
new file mode 100644
index 0000000..5ef3841
--- /dev/null
+++ b/bcc/misc/android-futex-contention.py
@@ -0,0 +1,115 @@
+# futex contention
+# (c) 2010, Arnaldo Carvalho de Melo <acme@redhat.com>
+# Licensed under the terms of the GNU GPL License version 2
+#
+# Translation of:
+#
+# http://sourceware.org/systemtap/wiki/WSFutexContention
+#
+# to perf python scripting.
+#
+# Measures futex contention
+
+import os, sys
+sys.path.append(os.environ['PERF_EXEC_PATH'] + '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
+from Util import *
+
+process_names = {}
+thread_thislock = {}
+thread_blocktime = {}
+
+lock_waits = {} # long-lived stats on (tid,lock) blockage elapsed time
+waker_wakee = {} # maps the futex waker to wakee
+max_waits = {} # Details about a maximum contention like owner, owner chain
+process_names = {} # long-lived pid-to-execname mapping
+
+def android_lock(callchain):
+ for c in callchain:
+ if 'sym' in c and 'name' in c['sym']:
+ name = c['sym']['name']
+ else:
+ continue
+
+ if 'art::Monitor::Lock' in name:
+ return True
+ return False
+
+def print_callchain(callchain):
+ for c in callchain:
+ if 'sym' in c and 'name' in c['sym']:
+ name = c['sym']['name']
+ else:
+ continue
+
+ print(" %s" % (name))
+
+def sched__sched_waking(event_name, context, common_cpu,
+ common_secs, common_nsecs, common_pid, common_comm,
+ common_callchain, comm, pid, prio, success,
+ target_cpu):
+ waker_wakee[pid] = [common_pid, common_callchain]
+
+def syscalls__sys_enter_futex(event, ctxt, cpu, s, ns, tid, comm, callchain,
+ nr, uaddr, op, val, utime, uaddr2, val3):
+
+ cmd = op & FUTEX_CMD_MASK
+ if cmd != FUTEX_WAIT or android_lock(callchain) == False:
+ return # we don't care about originators of WAKE events
+ # or futex uses that aren't android locks.
+
+ process_names[tid] = comm
+ thread_thislock[tid] = uaddr
+ thread_blocktime[tid] = nsecs(s, ns)
+
+def syscalls__sys_exit_futex(event, ctxt, cpu, s, ns, tid, comm, callchain,
+ nr, ret):
+
+ waker_pid = -1
+ waker_chain = "[no call chain]"
+
+ if thread_blocktime.has_key(tid):
+ # Gather stats about the contention (sum, min, max)
+ elapsed = nsecs(s, ns) - thread_blocktime[tid]
+ add_stats(lock_waits, (tid, thread_thislock[tid]), elapsed)
+
+ # Track details about the maximum contention seen
+ # including owner and its callchain
+ if (tid, thread_thislock[tid]) in max_waits:
+ prev_wait = max_waits[(tid, thread_thislock[tid])][0]
+ else:
+ prev_wait = 0
+
+ if elapsed > prev_wait:
+ if tid in waker_wakee:
+ waker_pid = waker_wakee[tid][0]
+ waker_chain = waker_wakee[tid][1]
+
+ max_waits[(tid, thread_thislock[tid])] = [elapsed, waker_pid, waker_chain, callchain]
+
+ del thread_blocktime[tid]
+ del thread_thislock[tid]
+
+def trace_begin():
+ print "Press control+C to stop and show the summary"
+
+def trace_end():
+ for (tid, lock) in lock_waits:
+ print("\n==============================================================\n")
+ min, max, avg, count = lock_waits[tid, lock]
+ print "%s[%d] lock %x contended %d times, %d avg ns, %d max ns" % \
+ (process_names[tid], tid, lock, count, avg, max)
+ print ""
+
+ if not (tid, lock) in max_waits:
+ print"Max contention info not available"
+ continue
+
+ print "Callstack of suffering task:"
+ print_callchain(max_waits[tid, lock][3])
+ print ""
+
+ waker_pid = max_waits[tid, lock][1]
+ waker_name = process_names[waker_pid] if waker_pid in process_names else "nameless-owner"
+ print "Owner %s caused this contention of %d ns. Owner's Call stack below:" % (waker_name, max_waits[tid, lock][0])
+ print_callchain(max_waits[tid, lock][2])
+
diff --git a/bcc/misc/futex-contention.py b/bcc/misc/futex-contention.py
new file mode 100644
index 0000000..5c96780
--- /dev/null
+++ b/bcc/misc/futex-contention.py
@@ -0,0 +1,63 @@
+# futex contention
+# (c) 2010, Arnaldo Carvalho de Melo <acme@redhat.com>
+# Licensed under the terms of the GNU GPL License version 2
+#
+# Translation of:
+#
+# http://sourceware.org/systemtap/wiki/WSFutexContention
+#
+# to perf python scripting.
+#
+# Measures futex contention
+
+import os, sys
+sys.path.append(os.environ['PERF_EXEC_PATH'] + '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
+from Util import *
+
+process_names = {}
+thread_thislock = {}
+thread_blocktime = {}
+
+lock_waits = {} # long-lived stats on (tid,lock) blockage elapsed time
+process_names = {} # long-lived pid-to-execname mapping
+
+def android_lock(callchain):
+ for c in callchain:
+ if 'sym' in c and 'name' in c['sym']:
+ name = c['sym']['name']
+ else:
+ continue
+
+ if 'art::Monitor::Lock' in name:
+ return True
+ return False
+
+def syscalls__sys_enter_futex(event, ctxt, cpu, s, ns, tid, comm, callchain,
+ nr, uaddr, op, val, utime, uaddr2, val3):
+
+ cmd = op & FUTEX_CMD_MASK
+ if cmd != FUTEX_WAIT or android_lock(callchain) == False:
+ return # we don't care about originators of WAKE events
+ # or futex uses that aren't android locks.
+
+ process_names[tid] = comm
+ thread_thislock[tid] = uaddr
+ thread_blocktime[tid] = nsecs(s, ns)
+
+def syscalls__sys_exit_futex(event, ctxt, cpu, s, ns, tid, comm, callchain,
+ nr, ret):
+ if thread_blocktime.has_key(tid):
+ elapsed = nsecs(s, ns) - thread_blocktime[tid]
+ add_stats(lock_waits, (tid, thread_thislock[tid]), elapsed)
+ del thread_blocktime[tid]
+ del thread_thislock[tid]
+
+def trace_begin():
+ print "Press control+C to stop and show the summary"
+
+def trace_end():
+ for (tid, lock) in lock_waits:
+ min, max, avg, count = lock_waits[tid, lock]
+ print "%s[%d] lock %x contended %d times, %d avg ns" % \
+ (process_names[tid], tid, lock, count, avg)
+
diff --git a/bcc/misc/lockstat.py b/bcc/misc/lockstat.py
new file mode 100644
index 0000000..fa3edda
--- /dev/null
+++ b/bcc/misc/lockstat.py
@@ -0,0 +1,300 @@
+#!/usr/bin/python
+#
+# lockstat Trace and display lock contention stats
+#
+# USAGE: lockstat
+
+# Licensed under the Apache License, Version 2.0 (the "License")
+# 28-Jul-2017 Gisle Dankel Created this.
+
+from bcc import BPF
+from ctypes import c_int
+from time import sleep
+from datetime import datetime
+import argparse
+import subprocess
+import os
+
+# One Lock object per TGID and uaddr.
+class Lock(object):
+ def __init__(self):
+ self.contention_count = 0
+ self.elapsed_blocked = 0
+ self.thread_count = 0
+ self.last_stack_syms = []
+
+ def update(self, count, block_time, last_stack_syms):
+ self.contention_count += count
+ self.elapsed_blocked += block_time
+ self.thread_count += 1
+ self.last_stack_syms = last_stack_syms
+
+def run_command_get_pid(command):
+ p = subprocess.Popen(command.split())
+ return p.pid
+
+examples = """
+EXAMPLES:
+
+./lockstat
+ Trace calls to sys_futex and display contented locks every 5 seconds
+ for all processes running on the system
+./lockstat -p <pid>
+ Trace only for the specified pid and display contended locks
+ every 5 seconds
+./lockstat -p <pid> -t
+ Trace for a specified pid and print a message on each entry and exit to
+ sys_futex until interrupted or killed
+./lockstat -p <pid> 10
+ Trace the specified pid and show a message every 10 seconds
+./lockstat -c <command> 1 30
+ Run the specified command and display contended locks every 1 second
+ for a total of 30 times
+"""
+
+description = """
+Trace kernel futex events.
+These often occur because of lock contention, e.g. involving a pthread_mutex.
+This script resemblers the following SystemTap example:
+https://sourceware.org/systemtap/SystemTap_Beginners_Guide/futexcontentionsect.html
+"""
+
+parser = argparse.ArgumentParser(description=description,
+ formatter_class=argparse.RawDescriptionHelpFormatter,
+ epilog=examples)
+parser.add_argument("-p", "--pid", type=int, default=-1,
+ help="the PID to trace; if not specified, trace all")
+parser.add_argument("-t", "--trace", action="store_true",
+ help="print trace messages for each futex enter/exit")
+parser.add_argument("interval", nargs="?", default=5, type=int,
+ help="interval in seconds to print summary")
+parser.add_argument("count", nargs="?", type=int,
+ help="number of times to print the report before exiting")
+parser.add_argument("-c", "--command",
+ help="execute and trace the specified command")
+
+args = parser.parse_args()
+
+pid = args.pid
+command = args.command
+interval = args.interval
+num_prints = args.count
+trace_all = args.trace
+
+if command is not None:
+ print("Executing '%s' and tracing the resulting process." % command)
+ pid = run_command_get_pid(command)
+
+bpf_source = """
+#include <uapi/linux/futex.h>
+#include <uapi/linux/ptrace.h>
+#include <linux/sched.h>
+#include <linux/time.h>
+
+struct comm_t {
+ char name[TASK_COMM_LEN];
+};
+
+struct lock_key_t {
+ u64 uaddr;
+ u32 pid;
+ u32 tgid;
+};
+
+struct lock_info_t {
+ u64 elapsed_blocked;
+ u64 contention_count;
+ u64 sid;
+};
+
+BPF_HASH(pid_lock, u32, u64);
+BPF_HASH(pid_blocktime, u32, u64);
+BPF_HASH(tgid_comm, u32, struct comm_t);
+BPF_HASH(lock_stats, struct lock_key_t, struct lock_info_t, 1000000);
+BPF_STACK_TRACE(stack_traces, 16384);
+
+static inline int update_stats(u32 pid, u32 tgid, u64 uaddr, u64 block_time, u64 sid) {
+ struct lock_key_t key = {};
+ struct lock_info_t zero = {};
+ struct lock_info_t *info;
+
+ key.pid = pid;
+ key.tgid = tgid;
+ key.uaddr = uaddr;
+ info = lock_stats.lookup_or_init(&key, &zero);
+ info->elapsed_blocked += block_time;
+ info->contention_count++;
+ info->sid = sid;
+
+ if (0 == tgid_comm.lookup(&tgid)) {
+ struct comm_t comm;
+ bpf_get_current_comm(&comm.name, sizeof(comm.name));
+ tgid_comm.update(&tgid, &comm);
+ }
+ return 0;
+}
+
+// FIXME: Should attach to sys_enter_futex and sys_exit_futex tracepoints here,
+// but that does not currently work
+int sys_futex_enter(struct pt_regs *ctx, u32 *uaddr, int op, u32 val,
+ struct timespec *utime, u32 *uaddr2, u32 val3) {
+ int cmd = op & FUTEX_CMD_MASK;
+ if (cmd != FUTEX_WAIT)
+ return 0;
+
+ u64 pid_tgid = bpf_get_current_pid_tgid();
+ u32 pid = pid_tgid;
+ u32 tgid = pid_tgid >> 32;
+
+ if (!(THREAD_FILTER))
+ return 0;
+
+ u64 timestamp = bpf_ktime_get_ns();
+ u64 uaddr64 = (u64) uaddr;
+ pid_lock.update(&pid, &uaddr64);
+ pid_blocktime.update(&pid, ×tamp);
+
+ if (SHOULD_PRINT)
+ bpf_trace_printk("enter sys_futex, pid = %u, uaddr = %x, "
+ "cmd = %u\\n", pid, uaddr64, cmd);
+ return 0;
+}
+
+int sys_futex_exit(struct pt_regs *ctx) {
+ u64 pid_tgid = bpf_get_current_pid_tgid();
+ u32 pid = pid_tgid;
+ u32 tgid = pid_tgid >> 32;
+ if (!(THREAD_FILTER))
+ return 0;
+
+ u64 *blocktime = pid_blocktime.lookup(&pid);
+ u64 *uaddr = pid_lock.lookup(&pid);
+ u64 timestamp = bpf_ktime_get_ns();
+ u64 elapsed;
+ u64 sid;
+
+ if (blocktime == 0 || uaddr == 0)
+ return 0; // not FUTEX_WAIT, or (less likely) missed futex_enter
+
+ elapsed = timestamp - *blocktime;
+
+ sid = stack_traces.get_stackid(ctx, BPF_F_USER_STACK);
+ update_stats(pid, tgid, *uaddr, elapsed, sid);
+ pid_lock.delete(&pid);
+ pid_blocktime.delete(&pid);
+
+ if (SHOULD_PRINT) {
+ bpf_trace_printk("exit sys_futex, uaddr = %x, elapsed = %uns\\n",
+ uaddr == 0 ? 0 : *uaddr, elapsed);
+ }
+ return 0;
+}
+
+"""
+
+bpf_source = bpf_source.replace("SHOULD_PRINT", "1" if trace_all else "0")
+
+thread_filter = '1'
+if pid != -1:
+ print("Tracing pid %d, Ctrl+C to quit." % pid)
+ # 'tgid' in kernel space is what people thin of as 'pid' in userspace
+ thread_filter = "tgid == %d" % pid
+else:
+ print("Tracing all processes, Ctrl+C to quit.")
+
+bpf_source = bpf_source.replace("THREAD_FILTER", thread_filter)
+
+bpf_program = BPF(text=bpf_source)
+bpf_program.attach_kprobe(event="SyS_futex", fn_name="sys_futex_enter")
+bpf_program.attach_kretprobe(event="SyS_futex", fn_name="sys_futex_exit")
+
+def get_syms(stack, pid):
+ global bpf_program
+ syms = []
+ for addr in stack:
+ s = bpf_program.sym(addr, pid, show_offset=True)
+ syms.append(s)
+ return syms
+
+def print_syms(syms):
+ print("=========")
+ for f in syms:
+ print(f)
+ print("=========")
+
+def is_android_monitor_lock(syms):
+ for s in syms:
+ if 'art::Monitor::Lock' in s:
+ return True
+ return False
+
+def disp_stack(stack, pid):
+ for addr in stack:
+ s = bpf_program.sym(addr, pid, show_offset=True)
+
+def create_tgid_stats():
+ global bpf_program
+ stats = bpf_program["lock_stats"]
+ res = {}
+ stack_traces = bpf_program['stack_traces']
+ for key, val in stats.items():
+ # Only display Android monitor locks
+ if val.sid >= 0:
+ ust = stack_traces.walk(val.sid)
+ syms = get_syms(ust, key.pid)
+ if not is_android_monitor_lock(syms):
+ continue
+ else:
+ continue
+
+ if not key.tgid in res:
+ res[key.tgid] = {}
+ if not key.uaddr in res[key.tgid]:
+ res[key.tgid][key.uaddr] = Lock()
+
+ lock = res[key.tgid][key.uaddr]
+ lock.update(val.contention_count, val.elapsed_blocked, syms)
+ return res
+
+def print_comm_stats(stats):
+ if stats == {}:
+ return
+
+ comms = bpf_program["tgid_comm"]
+ print("\n%s:" % (datetime.now().strftime("%H:%M:%S")))
+ for tgid, locks in stats.items():
+ comm = comms[c_int(tgid)].name
+ print("\n %s (%d):" % (comm, tgid))
+ sorted_locks = sorted(locks.items(),
+ key=lambda x: x[1].elapsed_blocked,
+ reverse=True)
+ for addr, stats in sorted_locks:
+ print(" %x: %dms (%d contentions involving %d threads, avg %dus)" %
+ (addr, stats.elapsed_blocked / 1000000,
+ stats.contention_count, stats.thread_count,
+ stats.elapsed_blocked / stats.contention_count / 1000))
+
+ # No use of displaying lock stacks since we're only
+ # filtering for Android monitor locks.
+ #
+ # print("Last stack for this lock:")
+ # print_syms(stats.last_stack_syms)
+
+count_so_far = 0
+while True:
+ if trace_all:
+ print(bpf_program.trace_fields())
+ else:
+ try:
+ sleep(interval)
+ except KeyboardInterrupt:
+ exit()
+ print_comm_stats(create_tgid_stats())
+ count_so_far += 1
+ bpf_program['tgid_comm'].clear()
+ bpf_program['lock_stats'].clear()
+ bpf_program['pid_lock'].clear()
+ bpf_program['pid_blocktime'].clear()
+
+ if num_prints is not None and count_so_far >= num_prints:
+ exit()
diff --git a/buildimage b/buildimage
new file mode 100755
index 0000000..ccdc65d
--- /dev/null
+++ b/buildimage
@@ -0,0 +1,11 @@
+#!/bin/bash
+set -e
+
+# Given a path, make an ext4 image out of it, store it in $2
+dd if=/dev/zero of=$2 bs=4k count=$((256 * 1024 * 2))
+mkfs.ext4 $2
+
+OUT=`mktemp -d`
+mount -o loop $2 $OUT/
+rsync -ra $1/ $OUT/
+umount $OUT/
diff --git a/buildstrap b/buildstrap
new file mode 100755
index 0000000..b33b71e
--- /dev/null
+++ b/buildstrap
@@ -0,0 +1,76 @@
+#!/bin/bash
+
+spath="$(dirname "$(readlink -f "$0")")"
+source $spath/utils/support
+source $spath/utils/banners
+
+ARCH=$1
+DISTRO=$2
+TDIR=$3
+OUT_TMP=$4
+PACKAGES=$5
+EXTRA_FILES="$(cat $6)"
+INSTALL_BCC=$7
+SKIP_DEVICE=$8 # Skip any device-specific stages
+VARIANT="--variant=minbase"
+
+time qemu-debootstrap --arch $ARCH --include=$PACKAGES $VARIANT \
+ $DISTRO $OUT_TMP http://deb.debian.org/debian/
+
+# Some reason debootstrap leaves these mounted
+umount $OUT_TMP/proc/sys/fs/binfmt_misc || true
+umount $OUT_TMP/proc || true
+
+# Make bash the default shell
+chroot $OUT_TMP rm /bin/sh || true
+chroot $OUT_TMP ln -s /bin/bash /bin/sh || true
+cp $spath/addons/bashrc $OUT_TMP/.bashrc
+cp $spath/addons/bashrc.common $OUT_TMP/.bashrc.common
+cp $spath/addons/bashrc.silent $OUT_TMP/.bashrc.silent
+cp $spath/addons/get_kvers.sh $OUT_TMP/
+
+for f in $EXTRA_FILES; do
+ if [ $f == "none" ]; then continue; fi
+ cp $f $OUT_TMP/
+done
+
+# Cleanup
+rm -rf $OUT_TMP/lib/udev/*
+rm -rf $OUT_TMP/var/lib/apt/lists/*
+rm -rf $OUT_TMP/var/cache/apt/archives/*deb
+rm -rf $OUT_TMP/usr/share/locale/*
+rm -rf $OUT_TMP/usr/lib/share/locale/*
+rm -rf $OUT_TMP/usr/share/doc/*
+rm -rf $OUT_TMP/usr/lib/share/doc/*
+rm -rf $OUT_TMP/usr/share/ieee-data/*
+rm -rf $OUT_TMP/usr/lib/share/ieee-data/*
+rm -rf $OUT_TMP/usr/share/man/*
+rm -rf $OUT_TMP/usr/lib/share/man/*
+
+# Fix apt-get issue: Android requires _apt user to be in the
+# AID_INET group which is also android specific.
+grep -ri _apt:x:100:65534 $OUT_TMP/etc/passwd > /dev/null 2>&1
+if [ $? -ne 0 ]; then
+ c_warning "_apt user cannot be added to AID_INET group"
+else
+ sed -i -e 's/_apt:x:100:65534/_apt:x:100:3003/' $OUT_TMP/etc/passwd
+fi
+
+# Add a default DNS server
+echo "nameserver 4.2.2.2" > $OUT_TMP/etc/resolv.conf
+
+# Clone BCC if needed
+if [ $INSTALL_BCC -eq 1 ]; then
+ git clone https://github.com/iovisor/bcc.git $TDIR/debian/bcc-master
+ cp $spath/bcc/build-bcc.sh $TDIR/debian/bcc-master/;
+fi
+
+# Should be really do this?
+chmod -R 0777 $TDIR/
+
+[ $SKIP_DEVICE -eq 0 ] || exit 0
+
+c_info "Compressing new filesystem to prepare to push to Android /data/androdeb/"
+tar -zcf $TDIR/deb.tar.gz -C $TDIR debian
+
+chmod 0777 $TDIR/deb.tar.gz
diff --git a/packages/bcc b/packages/bcc
new file mode 100644
index 0000000..1d91354
--- /dev/null
+++ b/packages/bcc
@@ -0,0 +1,25 @@
+PACKAGES+="
+llvm-6.0-dev
+libclang-6.0-dev
+libelf-dev
+libfl-dev
+libunwind-dev
+libdw-dev
+git
+gcc
+libtool
+autoconf
+make
+cmake
+iperf
+arping
+ethtool
+flex
+bison
+python
+clang-6.0
+python-netaddr
+python-pyroute2
+"
+
+INSTALL_BCC=1
diff --git a/packages/compilers b/packages/compilers
new file mode 100644
index 0000000..f380dd3
--- /dev/null
+++ b/packages/compilers
@@ -0,0 +1,9 @@
+PACKAGES+="\
+git
+clang-6.0
+gcc
+libtool
+autoconf
+make
+cmake
+"
diff --git a/packages/editors b/packages/editors
new file mode 100644
index 0000000..99865fa
--- /dev/null
+++ b/packages/editors
@@ -0,0 +1,5 @@
+PACKAGES+="\
+vim
+nano
+git
+"
diff --git a/packages/scheduler b/packages/scheduler
new file mode 100644
index 0000000..a5df38c
--- /dev/null
+++ b/packages/scheduler
@@ -0,0 +1,4 @@
+PACKAGES+="\
+git
+rt-app
+"
diff --git a/packages/tracers b/packages/tracers
new file mode 100644
index 0000000..38c1d43
--- /dev/null
+++ b/packages/tracers
@@ -0,0 +1,5 @@
+PACKAGES+="\
+linux-perf
+trace-cmd
+strace
+"
diff --git a/utils/banners b/utils/banners
new file mode 100755
index 0000000..5f975bc
--- /dev/null
+++ b/utils/banners
@@ -0,0 +1,59 @@
+#!/bin/bash -x
+
+print_prepare_banner() {
+ c_info "Preparing device..."
+ if [ $FULL -eq 1 ]; then
+ c_info "Doing a full install."
+ else
+ c_info "Doing a base install."
+ fi
+ c_info ""
+}
+
+usage() {
+ c_info "USAGE:"
+ c_info "adeb"
+ c_info " shell Enter the androdeb shell environment and get to work!"
+ c_info " remove Remove androdeb from the device"
+ c_info " git-pull Git pull androdeb to update it on your host"
+ c_info " pull Copy files from the androdeb filesystem in the device"
+ c_info " push Copy files to the androdeb filesystem in the device"
+ c_info ""
+ c_info " prepare Prepare the device (when running for the first time)"
+ c_info " By default, this will download and install a base image."
+ c_info " ** Folowing are the prepare options **"
+ c_info " --full Pass this to prepare to download and install the full image which"
+ c_info " contains compilers, editors, tracers etc."
+ c_info ""
+ c_info " --build Instead of download, build and install the image onto the device"
+ c_info ""
+ c_info " --archive Use archive for root fs (overrides all other prepare options)"
+ c_info ""
+ c_info " --buildtar While preparing, also build a tar.gz.zip file of the filesystem,"
+ c_info " this is how images that are downloaded by prepare are built"
+ c_info ""
+ c_info " --build-image Build an ext4 .img with the base image and BCC (useful for Qemu)"
+ c_info ""
+ c_info " ** Folowing are misc build options **"
+ c_info " --tempdir Use a specific temporary directory for build operation"
+ c_info " --arch Specify an ARCH to build for (default arm64)"
+ c_info " --distro Debian distro to base on (default is buster)"
+ c_info ""
+ c_info " ** Folowing are the options for BCC **"
+ c_info " --bcc Build and install BCC onto the device, from source"
+ c_info " BCC is already included in 'prepare --full'"
+ c_info ""
+ c_info " --kernelsrc Extract kernel headers for BCC from this directory"
+ c_info ""
+ c_info " --skip-install Pass this to --kernelsrc if you wish to only extra/install kernel headers"
+ c_info " and would like to exit after that (skips build/install of everything else)"
+ c_info ""
+ c_info " ** Folowing are device specific options ** "
+ c_info " --device Serial number of adb device."
+ c_info " -s Serial number of adb device."
+ c_info ""
+ c_info " --debug Debug all execution."
+ exit 1
+}
+
+
diff --git a/utils/packheaders.sh b/utils/packheaders.sh
new file mode 100755
index 0000000..d10b81b
--- /dev/null
+++ b/utils/packheaders.sh
@@ -0,0 +1,25 @@
+#!/bin/bash
+
+# Utility to build kernel headers tar/zip file
+# must be run from the top level of a kernel source directory
+# and supplied an output file name
+
+MKTEMP=0; if [[ -z ${TDIR+x} ]] || [[ ! -d "${TDIR}" ]]; then
+ TDIR=`mktemp -d`; MKTEMP=1; fi
+rm -rf $TDIR/*
+TDIR_ABS=$( cd "$TDIR" ; pwd -P )
+
+
+if [ $# -ne 1 ]; then
+ echo "usage: makeheaders.sh <output file name>"
+ exit 0
+fi
+
+mkdir -p $TDIR_ABS/kernel-headers
+
+find arch -name include -type d -print | xargs -n1 -i: find : -type f -exec cp --parents {} $TDIR_ABS/kernel-headers/ \;
+find include -exec cp --parents {} $TDIR_ABS/kernel-headers/ 2> /dev/null \;
+tar -zcf $1 --directory=$TDIR_ABS kernel-headers
+
+zip -r $1.zip $1
+rm -rf $TDIR/*; if [ $MKTEMP -eq 1 ]; then rm -rf $TDIR; fi
diff --git a/utils/support b/utils/support
new file mode 100755
index 0000000..ad72e05
--- /dev/null
+++ b/utils/support
@@ -0,0 +1,175 @@
+#!/bin/bash -x
+# Utilities to interact with android more easily
+
+function run_quiet() { eval "$* >/dev/null 2>&1"; }
+
+make_csv() {
+ out=""
+ in=$1
+ for p in $in; do
+ if [ "x$out" == "x" ]; then
+ out=$p
+ else
+ out="$out,$p"
+ fi
+ done
+ echo $out
+}
+
+make_spaces() {
+ out=""
+ in=$1
+ for p in $in; do
+ if [ "x$out" == "x" ]; then
+ out=$p
+ else
+ out="$out $p"
+ fi
+ done
+ echo $out
+}
+
+cmd_exists() {
+ which $1 > /dev/null
+ return $?
+}
+
+do_adb_root() {
+ ADB="$1"
+ $ADB root > /dev/null 2>&1
+ return $?
+}
+
+die() {
+ exit_code=$1
+ msg=$2
+ c_error "$msg"
+ exit $exit_code
+}
+
+die_if_no_androdeb() {
+ set +e
+ $ADB shell ls /data/androdeb/debian > /dev/null 2>&1
+ if [ $? -ne 0 ]; then die 8 "Existing androdeb env not found on device. $1"; fi
+ set -e
+}
+
+# Helper function to count number of source arguments in a list
+# when more than one argument is supplied, it is assumed the last argument
+# is a destination
+count_sources() {
+ local source_count=$#
+ if [ $source_count -gt 1 ]; then
+ source_count=$((source_count - 1))
+ fi
+ echo "$source_count"
+}
+
+# Borrowed from project LISA.
+################################################################################
+# Logging functions
+################################################################################
+c_error() {
+ NOW=$(date +"%H:%m:%S")
+ # If there is only one parameter, let's assume it's just the message
+ if [ $# -gt 1 ]; then
+ local parent_lineno="$1"
+ local message="$2"
+ echo -e "${red}$NOW - ERROR: on or near line ${parent_lineno}: ${message}${nocol}"
+ return
+ fi
+
+ local message="$1"
+ echo -e "${red}$NOW - ERROR : ${message}${nocol}"
+}
+
+c_warning() {
+ NOW=$(date +"%H:%m:%S")
+ # If there is only one parameter, let's assume it's just the message
+ if [ $# -gt 1 ]; then
+ local parent_lineno="$1"
+ local message="$2"
+ echo -e "${yellow}$NOW - WARNING: on or near line ${parent_lineno}: ${message}${nocol}"
+ return
+ fi
+ local message="$1"
+ echo -e "${yellow}$NOW - WARNING : ${message}${nocol}"
+}
+
+c_info() {
+ NOW=$(date +"%H:%m:%S")
+ # If there is only one parameter, let's assume it's just the message
+ if [ $# -gt 1 ]; then
+ local parent_lineno="$1"
+ local message="$2"
+ echo -e "${blue}$NOW - INFO: on or near line ${parent_lineno}: ${message}${nocol}"
+ return
+ fi
+ local message="$1"
+ echo -e "${blue}$NOW - INFO : ${message}${nocol}"
+}
+
+d_notify() {
+ MESSAGE=$1
+ ICON=$2
+ # Let's try to send a desktop notification,
+ # silently fails if there is not support.
+ notify-send \
+ --icon=$ICON \
+ --urgency=critical \
+ --expire-time=1500 \
+ "Test Series" \
+ "$MESSAGE" \
+ 2>/dev/null
+}
+
+my_tput() {
+ if [ "${TERM-dumb}" == dumb ]; then
+ return
+ fi
+ tput $*
+}
+
+box_out()
+{
+ local s=("$@") b w
+ for l in "${s[@]}"; do
+ ((w<${#l})) && { b="$l"; w="${#l}"; }
+ done
+ my_tput setaf 3
+ echo -e "|-${b//?/-}-|"
+ for l in "${s[@]}"; do
+ printf '| %s%*s%s |\n' "$(my_tput setaf 4)" "-$w" "$l" "$(my_tput setaf 3)"
+ # echo "|-${b//?/-}-|"
+ done
+ echo "|-${b//?/-}-|"
+ my_tput sgr 0
+}
+
+
+################################################################################
+# Colors
+################################################################################
+
+if [ -t 1 ]; then
+ ncolors=$(my_tput colors)
+ if [ -n "${ncolors}" ] && [ ${ncolors} -ge 8 ]; then
+ nocol='\e[0m' # No Color
+ white='\e[1;37m'
+ black='\e[0;30m'
+ blue='\e[0;34m'
+ lblue='\e[1;34m'
+ green='\e[0;32m'
+ lgreen='\e[1;32m'
+ cyan='\e[0;36m'
+ lcyan='\e[1;36m'
+ red='\e[0;31m'
+ lred='\e[1;31m'
+ purple='\e[0;35m'
+ lpurple='\e[1;35m'
+ brown='\e[0;33m'
+ yellow='\e[1;33m'
+ grey='\e[0;30m'
+ lgrey='\e[0;37m'
+ fi
+fi
