$ export branch=master # currently, internal master is mainly supported.
$ mkdir ${branch}
$ cd ${branch}
$ repo init -b ${branch} -u persistent-https://googleplex-android.git.corp.google.com/platform/manifest
$ repo sync -j 32
Then to check the VTS project directory,
$ ls test/vts
Create your test project using:
test/vts/script/create-test-project.py --name <your project name> --dir <your project directory>
Project name is required to be UpperCamel case (trailing numbers are allowed), and test directory is a relative directory under test/vts/testcases.
For example, to create your test project HelloAndroid under directory test/vts/testcases/codelab/hello_android, use the following command:
test/vts/script/create-test-project.py --name HelloAndroid --dir codelab/hello_android
This will create Android.mk file, AndroidTest.xml, __init__.py, and populate parent directories with necessary packaging files.
The actual python test case file, which contains two simple hello world tests, is created under the specified project directory using the lower under score version of the project name.
In practice, use
-dir host/<your_project_name> if your project is for HAL (Hardware Abstraction Layer) or libraries
-dir kernel/<your_project_name> if your project is for kernel or kernel modules
$ vi test/vts/tools/vts-tradefed/res/config/vts-codelab.xml
Then edit its contents to:
<configuration description="Run VTS CodeLab Tests"> <include name="everything" /> <option name="plan" value="vts" /> <option name="compatibility:include-filter" value="<your project name>" /> <template-include name="reporters" default="basic-reporters" /> </configuration>
Multiple instances of compatibility:include-filter option can be added to include more tests under a test suite.
$ . build/envsetup.sh
or $ source ./build/envsetup.sh
To run the tests against physical devices,
$ lunch <your target> (e.g., bullhead-userdebug, angler-userdebug)
$ make vts -j 10
You may run your test using VTS TradeFed console. First, start VTS TradeFed console:
$ vts-tradefed
Then, use run command to start your test plan:
> run vts-codelab
If your test module is not in a test plan, you can specify test module to run directly:
> run vts -m <your test module name>
Instead of using TradeFed console, you may also run test directly through shall and make it faster by disabling system check and run on primary ABI only:
vts-tradefed run commandAndExit vts --skip-all-system-status-check --primary-abi-only --skip-preconditions --module <your test module name> -l INFO
The -l INFO argument in the end would allow VTS TradeFed print host log to terminal (you may want to set Scrollback lines to a higher value in your terminal's setting).
If your test case can violate some SELinux rules, please run:
host$ adb shell
target$ su
target$ setenforce 0
Tip: you may put the build command and test run command together:
make vts -j10 && vts-tradefed run commandAndExit vts --skip-all-system-status-check --primary-abi-only --skip-preconditions --module <your test module name> -l INFO
Pre-test file pushes from host to device can be configured for VtsFilePusher in AndroidTest.xml.
By default, AndroidText.xml pushes a group of files required to run VTS framework specified in test/vts/tools/vts-tradefed/res/push_groups/HidlHalTest.push. Individual file push can be defined with “push” option inside VtsFilePusher. Please refer to TradeFed for more detail.
Python module dependencies can be specified as “dep-module” option for VtsPythonVirtualenvPreparer in Android.xml. This will trigger the runner to install or update the modules using pip before running tests. VtsPythonVirtualenvPreparer will install a set of package including future, futures, enum, and protobuf by default. To add a dependency module, please add <option name="dep-module" value="<module name>" /> inside VtsPythonVirtualenvPreparer in ‘AndroidTest.xml’
Optionally, a .config file can be used to pass variables in json format to test case.
To add a .config file, create a .config file under your project directory using project name:
$ vi test/vts/testcases/host/<your project directiry>/<your project name>.config
Then edit its contents to:
{ <key_1>: <value_1>, <key_2>: <value_2> }
And in your test case python class, you can get the json value by using self.getUserParams method.
For example:
required_params = ["key_1"] self.getUserParams(required_params) logging.info("%s: %s", "key_1", self.key_1)
At last, add the following line to com.android.tradefed.testtype.VtsMultiDeviceTest class in AndroidTest.xml:
<option name="test-config-path" value="vts/testcases/<your project directiry>/<your project name>.config" />
Your config file will overwrite the following default json object defined at test/vts/tools/vts-tradefed/res/default/DefaultTestCase.config
{ "test_bed": [ { "name": "<your project name>", "AndroidDevice": "*" } ], "log_path": "/tmp/logs", "test_paths": ["./"] }
Once a new test case is added to one of the launched test suites, it is automatically executed in a test lab (e.g., using some common devices). The exact schedule, and the used branches and devices are all customizable.
Results from the test lab are automatically visible on the dashboard, but local runs may also be visible for debugging purposes if tests are run on a machine with access to the database service key file. To view local runs on the dashboard for debugging purposes, add the following to the end of the URL from the table summary page: ‘&unfiltered=’.
For notifications when test cases fail, go to the VTS dashboard and click the settings button in the bottom right of the main page. Search for and add the test suite to your favorites list to subscribe to notifications.