atest: Add atest developer markdown.
Bug: 73783175
Test: None
Change-Id: I60ec594a214c7361e51d8fec0b62a9d5b02cff4c
diff --git a/atest/README.md b/atest/README.md
index 12cc31d..56aef40 100644
--- a/atest/README.md
+++ b/atest/README.md
@@ -7,6 +7,9 @@
Importantly, when writing your test's build script file (Android.mk), make sure to include
the variable `LOCAL_COMPATIBILITY_SUITE`. A good default to use for it is `device-test`.
+Curious about how atest works? Want to add a feature but not sure where to begin?
+Just want to learn about the overall structure? [Go here.](./docs/atest_structure.md)
+
##### Table of Contents
1. [Environment Setup](#environment-setup)
2. [Basic Usage](#basic-usage)
diff --git a/atest/docs/atest_structure.md b/atest/docs/atest_structure.md
new file mode 100644
index 0000000..1ff7b90
--- /dev/null
+++ b/atest/docs/atest_structure.md
@@ -0,0 +1,116 @@
+# Atest Developer Guide
+
+You're here because you'd like to contribute to atest. To start off, we'll
+explain how atest is structured and where the major pieces live and what they
+do. If you're more interested in how to use atest, go to the [README](../README.md).
+
+##### Table of Contents
+1. [Overall Structure](#overall-structure)
+2. [Major Files and Dirs](#major-files-and-dirs)
+3. [Test Finders](#test-finders)
+4. [Test Runners](#test-runners)
+5. [Constants Override](#constants-override)
+
+## <a name="overall-structure">Overall Structure</a>
+
+Atest is primarily composed of 2 components: [test finders](#test-finders) and
+[test runners](#test-runners). At a high level, atest does the following:
+1. Parse args and verify environment
+2. Find test(s) based on user input
+3. Build test dependencies
+4. Run test(s)
+
+Let's walk through an example run and highlight what happens under the covers.
+
+> ```# atest hello_world_test```
+
+Atest will first check the environment is setup and then load up the
+module-info.json file (and build it if it's not detected or we want to rebuild
+it). That is a critical piece that atest depends on. Module-info.json contains a
+list of all modules in the android repo and some relevant info (e.g.
+compatibility_suite, auto_gen_config, etc) that is used during the test finding
+process. We create the results dir for our test runners to dump results in and
+proceed to the first juicy part of atest, finding tests.
+
+The tests specified by the user are passed into the ```CLITranslator``` to
+transform the user input into a ```TestInfo``` object that contains all of the
+required and optional bits used to run the test as how the user intended.
+Required info would be the test name, test dependencies, and the test runner
+used to run the test. Optional bits would be additional args for the test and
+method/class filters.
+
+Once ```TestInfo``` objects have been constructed for all the tests passed in
+by the user, all of the test dependencies are built. This step can by bypassed
+if the user specifies only _-t_ or _-i_.
+
+The final step is to run the tests which is where the test runners do their job.
+All of the ```TestInfo``` objects get passed into the ```test_runner_handler```
+which invokes each ```TestInfo``` specified test runner. In this specific case,
+the ```AtestTradefedTestRunner``` is used to kick off ```hello_world_test```.
+
+Read on to learn more about the classes mentioned.
+
+## <a name="major-files-and-dirs">Major Files and Dirs</a>
+
+Here is a list of major files and dirs that are important to point out:
+* ```atest.py``` - Main entry point.
+* ```cli_translator.py``` - Home of ```CLITranslator``` class. Translates the
+ user input into something the test runners can understand.
+* ```test_finder_handler.py``` - Module that collects all test finders,
+ determines which test finder methods to use and returns them for
+ ```CLITranslator``` to utilize.
+* ```test_finders/``` - Location of test finder classes. More details on test
+ finders [below](#test-finders).
+* ```test_finders/test_info.py``` - Module that defines ```TestInfo``` class.
+* ```test_runner_handler.py``` - Module that collects all test runners and
+ contains logic to determine what test runner to use for a particular
+ ```TestInfo```.
+* ```test_runners/``` - Location of test runner classes. More details on test
+ runners [below](#test-runners).
+* ```constants_default.py``` - Location of constant defaults. Need to override
+ some of these constants for your private repo? [Instructions below](#constants-override).
+
+## <a name="test-finders">Test Finders</a>
+
+Test finders are classes that host find methods. The find methods are called by
+atest to find tests in the android repo based on the user's input (path,
+filename, class, etc). Find methods will also find the corresponding test
+dependencies for the supplied test, translating it into a form that a test
+runner can understand, and specifying the test runner.
+
+For more details and instructions on how to create new test finders,
+[go here](./develop_test_finders.md)
+
+## <a name="test-runners">Test Runners</a>
+
+Test Runners are classes that execute the tests. They consume a ```TestInfo```
+and execute the test as specified.
+
+For more details and instructions on how to create new test runners, [go here](./develop_test_runners.md)
+
+## <a name="constants-override">Constants Override</a>
+
+You'd like to override some constants but not sure how? Override them with your
+own constants_override.py that lives in your own private repo.
+
+1. Create new ```constants_override.py``` (or whatever you'd like to name it) in
+ your own repo. It can live anywhere but just for example purposes, let's
+ specify the path to be ```<private repo>/path/to/constants_override/constants_override.py```.
+2. Add a ```vendorsetup.sh``` script in ```//vendor/<somewhere>``` to export the
+ path of ```constants_override.py``` base path into ```PYTHONPATH```.
+```bash
+# This file is executed by build/envsetup.sh
+_path_to_constants_override="$(gettop)/path/to/constants_override"
+if [[ ! $PYTHONPATH == *${_path_to_constants_override}* ]]; then
+ export PYTHONPATH=${_path_to_constants_override}:$PYTHONPATH
+fi
+```
+3. Try-except import ```constants_override``` in ```constants.py```.
+```python
+try:
+ from constants_override import *
+except ImportError:
+ pass
+```
+4. You're done! To pick up the override, rerun build/envsetup.sh to kick off the
+ vendorsetup.sh script.
diff --git a/atest/docs/develop_test_finders.md b/atest/docs/develop_test_finders.md
new file mode 100644
index 0000000..5235ef7
--- /dev/null
+++ b/atest/docs/develop_test_finders.md
@@ -0,0 +1,64 @@
+# Test Finder Developer Guide
+
+Learn about test finders and how to create a new test finder class.
+
+##### Table of Contents
+1. [Test Finder Details](#test-finder-details)
+2. [Creating a Test Finder](#creating-a-test-finder)
+
+## <a name="test-finder-details">Test Finder Details</a>
+
+A test finder class holds find methods. A find method is given a string (the
+user input) and should try to resolve that string into a ```TestInfo``` object.
+A ```TestInfo``` object holds the test name, test dependencies, test runner, and
+a data field to hold misc bits like filters and extra args for the test. The
+test finder class can hold multiple find methods. The find methods are grouped
+together in a class so they can share metadata for optimal test finding.
+Examples of metadata would be the ```ModuleInfo``` object or the dirs that hold
+the test configs for the ```TFIntegrationFinder```.
+
+**When should I create a new test finder class?**
+
+If the metadata used to find a test is unlike existing test finder classes,
+that is the right time to create a new class. Metadata can be anything like
+file name patterns, a special file in a dir to indicate it's a test, etc. The
+default test finder classes use the module-info.json and specific dir paths
+metadata (```ModuleFinder``` and ```TFIntegrationFinder``` respectively).
+
+## <a name="creating-a-test-finder">Creating a Test Finder</a>
+
+First thing to choose is where to put the test finder. This will primarily
+depend on if the test finder will be public or private. If public,
+```test_finders/``` is the default location.
+
+> If it will be private, then you can
+> follow the same instructions for ```vendorsetup.sh``` in
+> [constants override](atest_structure.md#constants-override) where you will
+> add the path of where the test finder lives into ```$PYTHONPATH```. Same
+> rules apply, rerun ```build/envsetup.sh``` to update ```$PYTHONPATH```.
+
+Now define your class and decorate it with the
+```test_finder_base.find_method_register``` decorator. This decorator will
+create a list of find methods that ```test_finder_handler``` will use to collect
+the find methods from your test finder class. Take a look at
+```test_finders/example_test_finder.py``` as an example.
+
+Define the find methods in your test finder class. These find methods must
+return a ```TestInfo``` object. Extra bits of info can be stored in the data
+field as a dict. Check out ```ExampleFinder``` to see how the data field is
+used.
+
+Decorate each find method with the ```test_finder_base.register``` decorator.
+This is used by the class decorator to identify the find methods of the class.
+
+Final bit is to add your test finder class to ```test_finder_handler```.
+Try-except import it in the ```_get_test_finders``` method and that should be
+it. The find methods will be collected and executed before the default find
+methods.
+```python
+try:
+ from test_finders import new_test_finder
+ test_finders_list.add(new_test_finder.NewTestFinder)
+except ImportError:
+ pass
+```
diff --git a/atest/docs/develop_test_runners.md b/atest/docs/develop_test_runners.md
new file mode 100644
index 0000000..80388ac
--- /dev/null
+++ b/atest/docs/develop_test_runners.md
@@ -0,0 +1,64 @@
+# Test Runner Developer Guide
+
+Learn about test runners and how to create a new test runner class.
+
+##### Table of Contents
+1. [Test Runner Details](#test-runner-details)
+2. [Creating a Test Runner](#creating-a-test-runner)
+
+## <a name="test-runner-details">Test Runner Details</a>
+
+The test runner class is responsible for test execution. Its primary logic
+involve construction of the commandline given a ```TestInfo``` and
+```extra_args``` passed into the ```run_tests``` method. The extra_args are
+top-level args consumed by atest passed onto the test runner. It is up to the
+test runner to translate those args into the specific args the test runner
+accepts. In this way, you can think of the test runner as a translator between
+the atest CLI and your test runner's CLI. The reason for this is so that atest
+can have a consistent CLI for args instead of requiring the users to remember
+the differing CLIs of various test runners. The test runner should also
+determine its specific dependencies that need to be built prior to any test
+execution.
+
+## <a name="creating-a-test-runner">Creating a Test Runner</a>
+
+First thing to choose is where to put the test runner. This will primarily
+depend on if the test runner will be public or private. If public,
+```test_runners/``` is the default location.
+
+> If it will be private, then you can
+> follow the same instructions for ```vendorsetup.sh``` in
+> [constants override](atest_structure.md#constants-override) where you will
+> add the path of where the test runner lives into ```$PYTHONPATH```. Same
+> rules apply, rerun ```build/envsetup.sh``` to update ```$PYTHONPATH```.
+
+To create a new test runner, create a new class that inherits
+```TestRunnerBase```. Take a look at ```test_runners/example_test_runner.py```
+to see what a simple test runner will look like.
+
+**Important Notes**
+You'll need to override the following parent methods:
+* ```host_env_check()```: Check if host environment is properly setup for the
+ test runner. Raise an expception if not.
+* ```get_test_runner_build_reqs()```: Return a set of build targets that need
+ to be built prior to test execution.
+* ```run_tests()```: Execute the test(s).
+
+And define the following class vars:
+* ```NAME```: Unique name of the test runner.
+* ```EXECUTABLE```: Test runner command, should be an absolute path if the
+ command can not be found in ```$PATH```.
+
+There is a parent helper method (```run```) that should be used to execute the
+actual test command.
+
+Once the test runner class is created, you'll need to add it in
+```test_runner_handler``` so that atest is aware of it. Try-except import the
+test runner in ```_get_test_runners``` like how ```ExampleTestRunner``` is.
+```python
+try:
+ from test_runners import new_test_runner
+ test_runners_dict[new_test_runner.NewTestRunner.NAME] = new_test_runner.NewTestRunner
+except ImportError:
+ pass
+```
