Docs: Adding AfW Test Harness page + nav
Adding comments from CL and doc
Changing platform variable to linux-86
Changing linux-86 to linux-x86
Adding editorial comments
Adding new content for b/25790522
Adding new comments
Bug: 30389619, 25790522
Change-Id: Ia48c25728af390957631cff2d096d776e5038c4f
diff --git a/src/devices/devices_toc.cs b/src/devices/devices_toc.cs
index 9674d2b..eb00758 100644
--- a/src/devices/devices_toc.cs
+++ b/src/devices/devices_toc.cs
@@ -383,7 +383,8 @@
<li><a href="<?cs var:toroot ?>devices/tech/admin/provision.html">Provisioning</a></li>
<li><a href="<?cs var:toroot ?>devices/tech/admin/multiuser-apps.html">Multiuser Apps</a></li>
<li><a href="<?cs var:toroot ?>devices/tech/admin/enterprise-telephony.html">Enterprise Telephony</a></li>
- <li><a href="<?cs var:toroot ?>devices/tech/admin/testing-setup.html">Testing Setup</a></li>
+ <li><a href="<?cs var:toroot ?>devices/tech/admin/testing-provision.html">Testing Device Provisioning</a></li>
+ <li><a href="<?cs var:toroot ?>devices/tech/admin/testing-setup.html">Testing Device Administration</a></li>
</ul>
</li>
diff --git a/src/devices/tech/admin/provision.jd b/src/devices/tech/admin/provision.jd
index abd3e01..b69e3cf 100644
--- a/src/devices/tech/admin/provision.jd
+++ b/src/devices/tech/admin/provision.jd
@@ -97,17 +97,20 @@
<pre>adb shell am start -a android.app.action.PROVISION_MANAGED_PROFILE \
-c android.intent.category.DEFAULT \
- -e wifiSsid $(printf '%q' \"GoogleGuest\") \
+ -e wifiSsid $(printf '%q' \"WifiSSID\") \
-e deviceAdminPackage "com.google.android.deviceadminsample" \
-e android.app.extra.deviceAdminPackageName $(printf '%q'
.DeviceAdminSample\$DeviceAdminSampleReceiver) \
-e android.app.extra.DEFAULT_MANAGED_PROFILE_NAME "My Organisation"
</pre>
-<h2 id=device_owner_provisioning_via_nfc>Device owner provisioning via NFC</h2>
+<h2 id=device_owner_provisioning_via_nfc>Device owner provisioning</h2>
+<p>Use one of the following methods to set up device owner (DO)
+provisioning.</p>
-<p>Device owner provisioning via NFC is similar to the profile owner method but
-requires more bootstrapping. To use this method,
+<h3 id=do_provision_nfc>Provisioning via NFC</h3>
+<p>DO provisioning via NFC is similar to the profile owner method but requires
+more bootstrapping. To use this method,
<a href="http://developer.android.com/guide/topics/connectivity/nfc/nfc.html">NFC
bump</a> the device during the initial setup step (i.e., first page of the setup
wizard). This low-touch flow configures Wi-Fi, installs the DPC, and sets the
@@ -132,14 +135,12 @@
<item>application/com.android.managedprovisioning</item>
</pre>
-<h2 id=device_owner_provisioning_with_activation_code>Device owner provisioning
-with activation code</h2>
-
-<p>Select <em>Add Work Account</em> from the setup/out-of-box experience (this
-triggers a lookup of the EMM from Android servers). The device installs the EMM
-app and starts provisioning flow. As an extra option, Android device
-administration supports the option of using email address with a six-digit
-activation code to bootstrap the process as part of setup.</p>
+<h3 id=do_provision_cs>Provisioning via Cloud Services</h2>
+<p>Device owner provisioning via cloud services includes the ability to
+provision a device in device owner mode during out-of-the-box setup. The device
+can collect credentials (or tokens) and use them to perform a lookup to a cloud
+service, which can then be used to initiate the device owner provisioning
+process.</p>
<h2 id=emm_benefits>EMM benefits</h2>
@@ -175,3 +176,9 @@
profile. For testing guidance, see
<a href="{@docRoot}devices/tech/admin/testing-setup.html">Setting up Device
Testing</a>.</p>
+
+<h2 id=automate>Automated provisioning testing</h2>
+<p>To automate the testing of enterprise provisioning processes, use
+the Android for Work (AfW) Test Harness. For details, see
+<a href="{@docRoot}devices/tech/admin/testing-provision.html">Testing Device
+Provisioning</a>.</p>
diff --git a/src/devices/tech/admin/testing-provision.jd b/src/devices/tech/admin/testing-provision.jd
new file mode 100644
index 0000000..dd66a3b
--- /dev/null
+++ b/src/devices/tech/admin/testing-provision.jd
@@ -0,0 +1,323 @@
+page.title=Testing Device Provisioning
+@jd:body
+
+<!--
+ Copyright 2016 The Android Open Source Project
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<div id="qv-wrapper">
+ <div id="qv">
+ <h2>In this document</h2>
+ <ol id="auto-toc">
+ </ol>
+ </div>
+</div>
+
+<p>The Android for Work (AfW) Test Harness is a test suite for validating the
+AfW compatibility of Android devices. It includes support apps, test cases,
+configuration files, and a test runner (<code>afw-test-tradefed</code>) built on
+<code>cts-tradefed</code>. You should setup and run the AfW Test Harness after
+completing <a href="{@docRoot}devices/tech/admin/provision.html">Provisioning
+for Device Administration</a>.</p>
+
+<p class=note><strong>Note:</strong> Building and running the AfW Test Harness
+is similar to building and running the Android
+<a href="http://source.android.com/compatibility/cts/index.html">Compatibility
+Test Suite (CTS)</a>.</p>
+
+<h2 id=setup_env>Setting up a development environment</h2>
+<p>The development environment for the AfW Test Harness is similar to Android
+OS. Follow the steps in
+<a href="{@docRoot}source/requirements.html">Requirements</a> to set up a
+development machine.</p>
+
+<h2 id=download_source>Downloading source code</h2>
+<p>Download the AfW Test Harness source code using the steps in
+<a href="{@docRoot}source/downloading.html">Downloading the Source</a>. The AfW
+Test Harness source code is in the <code>./test/AfwTestHarness</code> project.
+The branch name determines the version of AfW Test Harness to download (each
+Android platform has a separate version of AfW Test Harness). For Android 6.0,
+the branch name is <code>afw-test-harness-marshmallow-dev</code>. To initialize
+the repo and download source code for this branch, use:</p>
+
+<pre>
+$ mkdir WORKING_DIRECTORY
+$ cd WORKING_DIRECTORY
+$ git config --global user.name "Your Name"
+$ git config --global user.email "you@example.com"
+$ repo init -u https://android.googlesource.com/platform/manifest -b afw-test-harness-marshmallow-dev
+$ repo sync -j24
+</pre>
+
+<p>To check out the source code for a different version, specify the branch with
+the corresponding tag. Available branches include:</p>
+
+<table>
+<tr>
+<th>Branch Name</td>
+<th>Supported Android Platform</td>
+</tr>
+<tr>
+<td>afw-test-harness-marshmallow-dev</td>
+<td>Android 6.0</td>
+</tr>
+<tr>
+<td>afw-test-harness-1.5</td>
+<td>Android 6.0</td>
+</tr>
+</table>
+
+<p>Other dependency projects required to build the harness are also downloaded
+with the source code.</p>
+
+<h3 id=view_studio>Viewing in Android Studio</h3>
+<p>To view and edit AfW source code in Android Studio:</p>
+<ol>
+<li>Run the following commands
+<pre>
+$ make idegen
+$ development/tools/idegen/idegen.sh
+</pre>
+</li>
+<li>In Android Studio, open <code>android.ipr</code>.</li>
+</ol>
+
+<p>The AfW Test Harness source code is in <code>test/AfwTestHarness</code>.</p>
+
+<h2 id=config_harness>Configuring the AfW Test Harness</h2>
+<p>You can customize the harness by configuring
+<code>test/AfwTestHarness/afw-test.props</code>. To run the harness
+successfully, complete the following steps:</p>
+<ol>
+<li>Configure the Wi-Fi network in <code>afw-test.props</code> with the
+following properties:
+<pre>wifi_ssid
+wifi_password (optional)
+wifi_security_type (optional, available options are: NONE, WEP or WPA)
+</pre>
+</li>
+<li>Obtain at least one account from a domain that is bound to Test DPC as
+its device policy controller. Specify the details in <code>afw-test.props</code>
+with the following properties:
+<pre>
+work_account_username
+work_account_password
+</pre>
+<p>The AfW Test Harness uses Test DPC to test provisioning flows, so accounts
+<strong>must</strong> bind to Test DPC to run the test harness.</p>
+</li>
+</ol>
+
+<h2 id=build_harness>Building the AfW Test Harness</h2>
+<p>Initialize the build configuration using:</p>
+<pre>
+$ source build/envsetup.sh
+$ lunch
+</pre>
+
+<p>Select a device type and press <strong>Enter</strong>.</p>
+
+<p>Build the harness using:</p>
+<pre>$ make afw-test-harness -j32</pre>
+<p>This creates a directory (<code>out/host/linux-x86/afw-th/android-cts</code>)
+with all necessary binaries, configuration files, and tools to run the test
+harness. This directory is also zipped into a file
+(<code>out/host/linux-x86/afw-th/android-afw-test-harness.zip</code>)
+for distribution.</p>
+
+<h2 id=run_harness>Running the AfW Test Harness</h2>
+<p>Use the following steps to run the AfW Test Harness:</p>
+<ol>
+<li>In your build environment, launch the test runner using:
+<pre>$ afw-test-tradefed</pre>
+This starts the <code>cts-tf</code> console, loads test plans, test cases,
+and <code>afw-test.props</code> from
+<code>out/host/linux-x86/afw-th/android-cts</code>.</li>
+<li>From the unzipped folder of <code>android-afw-test-harness.zip</code>,
+launch the test runner using:
+<pre>$ cts-tf > ./android‐cts/tools/afw-test‐tradefed</pre>
+This loads test plans, test cases, and <code>afw-test.props</code> from
+<code>android-cts</code> directory. Ensure
+<code>./android‐cts/repository/testcases/afw-test.props</code> has the work
+account and Wi-Fi configuration.</li>
+
+<li>Run a test plan. Each test plan is an XML file that contains a set of test
+packages from the <code>AfwTestHarness/tests</code> test package directory.
+Common plans include:
+
+<ul>
+<li><code><strong>afw-userdebug-build</code></strong>. Contains all test
+packages that require a userdebug build.</li>
+<li><code><strong>afw-user-build</code></strong>. Runs on a user build but
+requires the test device to be set up properly, including completing the initial
+setup and enabling USB debugging.</li>
+</ul>
+
+<br>To run the test plan <code>afw-userdebug-build</code>, use:
+<pre>$ cts-tf > run cts --plan afw-userdebug-build</pre>
+To see all test plans, use the command <code>list plans</code>. To view plan
+definitions, refer to
+<code>out/host/linux-x86/afw-th/android-cts/repository/plans</code>.
+<br>
+</li>
+<li>Run a test package. To run a single test package, use
+<pre>$ cts-tf > run cts --package com.android.afwtest.NfcProvisioning
+</pre>
+To view all packages, use the command <code>list packages</code>. For more
+options, use the command <code>run cts --help</code>.</li>
+</ol>
+
+<h2 id=debug_harness>Debugging the AfW Test Harness</h2>
+<p>Run all commands in the afw-test-tradefed console (<code>cts-tf</code>),
+which you can launch by running <code>afw-test-tradefed</code>.</p>
+<ul>
+
+<li>Display more information with the <code>-l INFO</code> or <code>-l
+DEBUG</code> flags. Example:
+<pre>$ cts-tf > run cts ‐‐plan afw-userdebug-build -l DEBUG</pre></li>
+
+<li>Run the test harness on a specific device with the <code>-s</code> flag.
+Example:
+<pre>$ cts-tf > run cts ‐‐plan afw-userdebug-build -l DEBUG -s device_sn</pre>
+</li>
+
+<li>Run test harness on all connected devices with the
+<code>--all-devices</code> flag. Example:
+<pre>$ cts-tf > run cts ‐‐plan afw-userdebug-build -l DEBUG --all-devices</pre>
+</li>
+
+<li>View current running executions using <code>list invocations</code> or
+<code>l i</code>.</li>
+
+<li>View summary of past test executions using <code>list results</code> or
+<code>l r</code>.</li>
+
+<li>View other <code>list</code> commands using <code>help list</code>.</li>
+
+<li>Monitor real-time logcat with filter using <code>afwtest</code>, then open
+another terminal and start logcat using: <code>adb logcat | grep afwtest</code>.
+After a test completes:
+<ul>
+<li>View logs in
+<code>out/host/linux-x86/afw-th/android-cts/repository/logs/<em>start-time</em></code>.
+The full device logcat and host log (<code>afw-test-tradefed</code> logs) are
+saved in separate zip files.</li>
+
+<li>Find relevant information by searching the device logcat for
+<strong>afwtest</strong>. Example: <code>zless
+out/host/linux-x86/afw-th/android-cts/repository/logs/<em>start-time</em>/device_logcat_<em>random-number</em>.zip
+| grep afwtest</code></li>
+
+<li>To view the full afw-test-tradefed log, use: <code>zless
+out/host/linux-x86/afw-th/android-cts/repository/logs/<em>start-time</em>/host_log_<em>random-number</em>.zip</code>
+</li>
+</ul>
+</li>
+<li>A test package automates an AfW provisioning flow by going through UI pages
+and recording a navigation log in the device logcat file for each page.
+Example: <code>afwtest.AutomationDriver:
+Navigating:com.android.afwtest.uiautomator.pages.gms.AddAccountPage</code>
+<br>UI pages for test package
+<code>com.android.afwtest.NfcProvisioning</code> include:<ul>
+<li>
+<code>com.android.afwtest.uiautomator.pages.managedprovisioning.NfcProvisioningPage</code>
+</li>
+<li><code>com.android.afwtest.uiautomator.pages.PageSkipper</code></li>
+<li><code>com.android.afwtest.uiautomator.pages.LandingPage</code></li>
+</ul>
+</li>
+<li>If a test failed during the provisioning process, logcat contains an error
+similar to:
+<pre>TestRunner: java.lang.RuntimeException: Failed to load page: com.android.afwtest.uiautomator.pages.packageinstaller.DeviceAccessPage
+</pre>
+This is typically caused by errors in a previous UI page or the page that
+failed to load, so try to find other error messages in logcat before this error,
+then try to reproduce it manually following the provisioning flow.</li>
+<li>If a test package fails:
+<ul>
+<li>A screenshot is saved to
+<code>out/host/linux-x86/afw-th/android-cts/repository/logs/<em>start-time</em></code>
+using the following syntax:
+<code>screenshot-test_<em>test_class_full_name</em>_<em>test_case_name</em>-<em>random_number</em>.png</code>.
+This information is also logged in the host log.</li>
+<li>A bug report is saved to
+<code>out/host/linux-x86/afw-th/android-cts/repository/logs/<em>start-time</em></code>
+as:
+<code>bug-<em>test_class_full_name</em>_<em>test_case_name</em>-<em>random_number</em>.zip</code>.
+</li>
+</ul>
+</li>
+<li>After all test packages execute, a screenshot is taken and saved to
+<code>out/host/linux-x86/afw-th/android-cts/repository/logs/<em>start-time</em></code>
+as: <code>screenshot-<em>random_number</em>.png</code>.
+This information is also logged in the host log.</li>
+</ul>
+</li>
+</ul>
+
+<h2 id=faq>FAQ</h2>
+<p>For help with questions not answered below, contact
+<a href="mailto:afw-testharness-support@google.com">afw-testharness-support@google.com</a>.
+</p>
+
+<p><strong>Can I run test plan <code>afw-userdebug-build</code> on a device
+flashed with user build?</strong></p>
+<p><em>No. Test packages in the <code>afw-userdebug-build</code> plan factory
+reset the testing device before running the actual test flow and require
+<code>adb</code> debugging to be auto-enabled. With a user build,
+<code>adb</code> debugging can be enabled only by manually changing the
+setting in Developer options.</em></p>
+
+<p><strong>Can I run test plan <code>afw-user-build</code> on a device flashed
+with userdebug build?</strong></p>
+<p><em>Yes, but we recommend that you run this test plan on a user build.</em></p>
+
+<p><strong>Sometimes my test fails because UI loading takes too much time. How
+can I fix this?</strong></p><em>Configure the <code>timeout_size</code> setting
+in <code>./android-cts/repository/testcases/afw-test.props</code>. Valid
+settings are: S, M, L, XL, XXL.</em></p>
+
+<p><strong>The test package
+<code>com.android.afwtest.NfcProvisioning</code> (or
+<code>SuwDoProvisioning</code>) fails on my device because the installed initial
+setup (i.e. Setup Wizard) shows customized UI (such as Term & Conditions)
+after provisioning is complete. How can I skip this customized UI?</strong></p>
+<p><em>There should be minimal UI after the provisioning process. The test
+harness will automatically skip such UI if the UI has a button that has
+meaningful text or content description that contains any of the following words:
+Skip, Finish, Done, Accept, Agree, Next, Continue, or Proceed. Alternatively,
+you can define a button in <code>afw-test.props</code> to configure the test
+harness to skip your UI. Example:</em></p>
+<pre>
+oem_widgets=your_btn
+your_btn.text=your_customized_text
+your_btn.package=your_package
+your_btn.action=click
+</pre>
+<em><p>To define multiple widgets, separate using commas.</em></p>
+
+<p><strong>The test package
+<code>com.android.afwtest.NfcProvisioning</code> (or
+<code>SuwDoProvisioning</code>) failed and the last UI screen is "Verify your
+account." Why does this happen and how can I recover the testing device?
+</strong></p>
+<p><em>This failure occurs because the previous test package failed to clear
+Factory Reset Protection at the end of the test. You must manually enter the
+account to unlock the device.</em></p>
+
+<p><strong>My device needs more time to factory reset. Can I extend the factory
+reset timeout?</strong></p>
+<p><em>Yes. Configure the <code>factory_reset_timeout_min</code> setting in
+<code>afw-test.props</code>. Valid settings are in minutes; you can set to any
+number of minutes that works with your device.</em></p>
diff --git a/src/devices/tech/admin/testing-setup.jd b/src/devices/tech/admin/testing-setup.jd
index 129fa8e..ea7a015 100644
--- a/src/devices/tech/admin/testing-setup.jd
+++ b/src/devices/tech/admin/testing-setup.jd
@@ -1,4 +1,4 @@
-page.title=Setting up Device Testing
+page.title=Testing Device Administration
@jd:body
<!--
@@ -72,6 +72,12 @@
Security > Device Administrators</em> and confirm TestDPC is in the
list. Verify it cannot be disabled (this signifies it is a device owner).</p>
+<h2 id=automate>Automated provisioning testing</h2>
+<p>To automate the testing of enterprise provisioning processes, use
+the Android for Work (AfW) Test Harness. For details, see
+<a href="{@docRoot}devices/tech/admin/testing-provision.html">Testing Device
+Provisioning</a>.</p>
+
<h2 id="troubleshooting">Bug reports and logs</h2>
<p>In Android N, device owner Device Policy Client (DPCs) can get bug reports
and view logs for enterprise processes on a managed device.</p>
