Docs: Update for CTS v2
Test: make online-sac-docs, staged on instance #28.
Bug: 30466328
Changes to the development.jd file (b/32324799) coming in a
separate CL.
Change-Id: If6129beae63cdbd64d3ea913621f8fc3ca34fca0
diff --git a/src/compatibility/cts/images/cts-v2-test-report.png b/src/compatibility/cts/images/cts-v2-test-report.png
new file mode 100644
index 0000000..3973922
--- /dev/null
+++ b/src/compatibility/cts/images/cts-v2-test-report.png
Binary files differ
diff --git a/src/compatibility/cts/images/cts-v2-test-summary.png b/src/compatibility/cts/images/cts-v2-test-summary.png
new file mode 100644
index 0000000..1910a6a
--- /dev/null
+++ b/src/compatibility/cts/images/cts-v2-test-summary.png
Binary files differ
diff --git a/src/compatibility/cts/interpret.jd b/src/compatibility/cts/interpret.jd
index 19df62e..d32b923 100644
--- a/src/compatibility/cts/interpret.jd
+++ b/src/compatibility/cts/interpret.jd
@@ -19,7 +19,7 @@
<p>The CTS test results are placed in the file:</p>
<pre>
-$CTS_ROOT/android-cts/repository/results/<start_time>.zip
+$CTS_ROOT/android-cts/repository/results/<start_time>.zip
</pre>
<p>If you have built the CTS yourself, <em>$CTS_ROOT</em> will resemble the
@@ -27,40 +27,74 @@
path where you have uncompressed the prebuilt official CTS <a
href="downloads.html">downloaded</a> from this site.</p>
-<p>Inside the zip, the testResult.xml file contains the actual results—open
+<p>Inside the zip, the testResult.xml file contains the actual results. Open
this file in any web browser (HTML5 compatible browser recommended) to view the
-test results. It will resemble the following screenshots.</p>
-
-<p class="note"><strong>Note:</strong> The results are provided to help you
-ensure the software remains compatible throughout the development process and
-act as a common format for communicating the compatibility status of your
-device with other parties.</p>
-
-<img src="images/cts-test-summary.png" alt="CTS test summary" id="figure1" />
-<p class="img-caption">
- <strong>Figure 1.</strong> CTS test summary
-</p>
+test results.</p>
<p>If testResult.xml displays a blank page when using the Chrome browser,
<a href="https://www.chromium.org/developers/how-tos/run-chromium-with-flags">change
your browser configuration</a> to enable the
<em>--allow-file-access-from-files</em> command line flag.</p>
-<p>The Device Information section provides details about the device, firmware
-(make, model, firmware build, platform), and device hardware (screen
-resolution, keypad, screen type). To access device information, click the link
-above Test Summary.</p>
+<h3 id="reading-the-test-results">Reading the test results</h3>
+
+<p>The details of the test results depend on which version of CTS you are
+using:</p>
+
+<ul>
+ <li>CTS v1 for Android 6.0 and earlier</a>
+ <li>CTS v2 for Android 7.0 and later</a>
+</ul>
+
+<p class="note"><strong>Note:</strong> The results are provided to help you
+ensure the software remains compatible throughout the development process and
+act as a common format for communicating the compatibility status of your
+device with other parties.</p>
+
+<h4 id="device-information">Device Information</h4>
+
+<p>In CTS v6.0 and earlier, select Device Information (link above Test Summary) to
+view details about the device, firmware (make, model, firmware build,
+platform), and device hardware (screen resolution, keypad, screen type). (CTS
+v7.0 does not display device information.)</p>
+
+<h4 id="test-summary">Test Summary</h4>
<p>The <em>Test Summary</em> section provides executed test plan details, like
the CTS plan name and execution start and end times. It also presents an
aggregate summary of the number of tests that passed, failed, timed out, or
could not be executed.</p>
-<p>The next section also provides a summary of tests passed per package.</p>
-<img src="images/cts-test-report.png" alt="CTS detailed test report" id="figure2" />
-<p class="img-caption">
- <strong>Figure 2.</strong> CTS test report
-</p>
+<div class="toggle-content closed">
+ <p><a href="#" onclick="return toggleContent(this)">
+ <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
+ <strong><span class="toggle-content-text">Show CTS v1 sample test summary</span>
+ <span class="toggle-content-text" style="display:none;">Hide CTS v1 sample test summary</span></strong>
+ </a></p>
+ <div class="toggle-content-toggleme">
+ <img src="images/cts-test-summary.png" alt="CTS v1 test summary" id="figure1a" />
+ <p class="img-caption"><strong>Figure 1a.</strong> CTS v1 sample test summary </p>
+ </div>
+</div>
+
+
+<div class="toggle-content closed">
+ <p><a href="#" onclick="return toggleContent(this)">
+ <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
+ <strong><span class="toggle-content-text">Show CTS v2 sample test summary</span>
+ <span class="toggle-content-text" style="display:none;">Hide CTS v2 sample test summary</span></strong>
+ </a></p>
+ <div class="toggle-content-toggleme">
+ <img src="images/cts-v2-test-summary.png" alt="CTS v2 test summary" id="figure1b" />
+ <p class="img-caption"><strong>Figure 1b.</strong> CTS v2 sample test summary </p>
+ </div>
+</div>
+
+<p> </p>
+
+
+<h4 id="test-report">Test Report</h4>
+<p>The next section, the CTS test report, provides a summary of tests passed per package.</p>
<p>This is followed by details of the the actual tests that were executed. The
report lists the test package, test suite, test case, and the executed tests.
@@ -73,3 +107,40 @@
editor should provide details of the test failure (search for the
<em><Test></em> tag corresponding to the failed test and look within it
for the <em><StackTrace></em> tag).</p>
+
+<div class="toggle-content closed">
+ <p><a href="#" onclick="return toggleContent(this)">
+ <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
+ <strong><span class="toggle-content-text">Show CTS v1 sample test report</span>
+ <span class="toggle-content-text" style="display:none;">Hide CTS v1 sample test report</span></strong>
+ </a></p>
+ <div class="toggle-content-toggleme">
+ <img src="images/cts-test-report.png" alt="CTS v1 test report" id="figure2a" />
+ <p class="img-caption"><strong>Figure 2a.</strong> CTS v1 sample test report</p>
+ </div>
+</div>
+
+<div class="toggle-content closed">
+ <p><a href="#" onclick="return toggleContent(this)">
+ <img src="{@docRoot}assets/images/triangle-closed.png" class="toggle-content-img" />
+ <strong><span class="toggle-content-text">Show CTS v2 sample test report</span>
+ <span class="toggle-content-text" style="display:none;">Hide CTS v2 sample test report</span></strong>
+ </a></p>
+ <div class="toggle-content-toggleme">
+ <img src="images/cts-v2-test-report.png" alt="CTS v2 test report" id="figure2b" />
+ <p class="img-caption"><strong>Figure 2b.</strong> CTS v2 sample test report</p>
+ </div>
+</div>
+
+<p> </p>
+
+<h3 id="incomplete-test-modules">Reviewing test_result.xml for incomplete test modules</h3>
+
+<p>
+To determine the number of incomplete modules in a given test session, run
+command 'list results'. The count of Modules Completed and Total Modules are
+listed for each previous session. To determine which modules are complete vs.
+incomplete, open the test_result.xml file and read the value of the "done"
+attribute for each module in the result report. Modules with value done =
+"false" have not run to completion.
+</p>
diff --git a/src/compatibility/cts/run.jd b/src/compatibility/cts/run.jd
index 9f8ffa5..9ecfe9d 100644
--- a/src/compatibility/cts/run.jd
+++ b/src/compatibility/cts/run.jd
@@ -43,14 +43,17 @@
<li>Launch the CTS console by running the <em>cts-tradefed</em> script from
the folder where the CTS package has been unzipped, e.g.
<code>$ ./android-cts/tools/cts-tradefed</code>
- <li>You may start the default test plan (containing all of the test packages) by
- appending: <code>run cts --plan CTS</code> (or just <code>run cts</code> for v2).
- This kicks off all the CTS tests required for compatibility.
- Enter <code>list plans</code> to see a list of test plans in the repository.
- Enter <code>list packages</code> to see a list of test packages in the repository.
- See the CTS command
- reference or type help for a complete list of supported commands.
- <li>Alternately, you may run the CTS plan of your choosing from the command line
+ <li>Start the default test plan (contains all test packages) by appending: <code>run
+ cts --plan CTS</code> . This kicks off all CTS tests required for compatibility.
+ <ul>
+ <li>For CTS v1 (Android 6.0 and earlier), enter <code>list plans</code> to view a list of
+ test plans in the repository or <code>list package</code>s to view a list of
+ test packages in the repository.
+ <li>For CTS v2 (Android 7.0 and later), enter <code>list modules</code> to
+ see a list of test modules.
+ </ul>
+ </li>
+ <li>Alternately, run the CTS plan of your choosing from the command line
using: <code>cts-tradefed run cts --plan
<plan_name>
</code>
diff --git a/src/compatibility/cts/setup.jd b/src/compatibility/cts/setup.jd
index c72de2d..169526c 100644
--- a/src/compatibility/cts/setup.jd
+++ b/src/compatibility/cts/setup.jd
@@ -77,8 +77,18 @@
directory name are correct.</p>
<h3 id=JDK>Java Development Kit (JDK)</h3>
-<p>Install the proper version of the Java Development Kit (JDK). See the <a
-href="{@docRoot}source/requirements.html#jdk">JDK requirements</a>.</p>
+<p>Install the proper version of the Java Development Kit (JDK). For Android 7.0—
+</p>
+<ul>
+ <li>On Ubuntu, use <a href="http://openjdk.java.net/install/">OpenJDK 8</a>.
+ <li>On Mac OS, use <a
+ href="http://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html#jdk-8u45-oth-JPR">jdk
+ 8u45 or newer</a>.</li>
+</ul>
+<p>
+For details, see the <a href="https://source.android.com/source/requirements.html#jdk">JDK
+requirements</a>.
+</p>
<h3 id=CTS_files>CTS files</h3>
@@ -129,6 +139,27 @@
ro.product.first_api_level=21
</pre>
+<h3 id=cts-shim-apps>CTS Shim apps</h3>
+
+<p> Android 7.0 includes the following pre-built apps (built from <a
+href="https://android.googlesource.com/platform/frameworks/base/+/master/packages/CtsShim/build/">this
+source</a>) which do not contain any code except for the manifest: </p>
+
+<ul>
+ <li><code><a href="https://android.googlesource.com/platform/frameworks/base/+/android-7.0.0_r1/packages/CtsShim/CtsShim.apk">
+ frameworks/base/packages/CtsShim/CtsShim.apk</a></code><br>
+ This apk file is copied to <code>/system/app/CtsShimPrebuilt.apk</code>
+ on the system image.
+ <li><code><a href="https://android.googlesource.com/platform/frameworks/base/+/android-7.0.0_r1/packages/CtsShim/CtsShimPriv.apk">
+ frameworks/base/packages/CtsShim/CtsShimPriv.apk</a></code><br>
+ This apk file is copied to <code>/system/priv-app/CtsShimPrivPrebuilt.apk</code>
+ on the system image.</li>
+</ul>
+
+<p> CTS uses these apps to test privileges and permissions. To pass the tests, you must
+preload the apps into the appropriate directories on the system image without
+re-signing them.</p>
+
<h3 id=storage_requirements>Storage requirements</h3>
<p>The CTS media stress tests require video clips to be on external storage
(<code>/sdcard</code>). Most of the clips are from <a
@@ -186,13 +217,14 @@
to <strong>Settings > About phone</strong> and tap <strong>Build number</strong>
seven times. Return to the previous screen to find <strong>Developer
options</strong>. See <a
- href="http://developer.android.com/tools/device.html#developer-device-options">Enabling
+ href="http://developer.android.com/studio/run/device.html#developer-device-options">Enabling
On-device Developer Options</a> for additional details.</p>
+ <li>Make sure the time is set to 12-hour format: <strong>Settings > Date & time > Use 24-hour format > Off</strong>
<li>Select: <strong>Settings > Developer options > Stay Awake > On</strong>
<li>Select: <strong>Settings > Developer options > Allow mock locations > On</strong>
-<p class="note"><strong>Note:</strong> This mock locations setting is applicable only in Android 5.x and 4.4.x.</p>
+ <p class="note"><strong>Note:</strong> This mock locations setting is applicable only in Android 5.x and 4.4.x.</p>
<li>Select: <strong>Settings > Developer options > Verify apps over USB > Off</strong>
-<p class="note"><strong>Note:</strong> This verify apps step became required in Android 4.2.</p>
+ <p class="note"><strong>Note:</strong> This verify apps step became required in Android 4.2.</p>
<li>Launch the browser and dismiss any startup/setup screen.
<li>Connect the desktop machine that will be used to test the device with a USB cable
<p class="note"><strong>Note:</strong> When you connect a device running Android 4.2.2 or later
@@ -204,18 +236,19 @@
<code>adb install -r android-cts/repository/testcases/CtsDelegatingAccessibilityService.apk</code><br>
On the device, enable: <strong>Settings > Accessibility > Accessibility >
Delegating Accessibility Service</strong></p>
-<p class="note"><strong>Note:</strong> For CTS 2.3 R4 through CTS 6.0 on devices that declare the
-<code>android.software.device_admin feature</code>, set up your device to run
-the device administration tests with:<br>
+<p class="note"><strong>Note:</strong> For CTS versions prior to 7.0, on devices that declare
+<code>android.software.device_admin</code>, set up your device to run
+the device administration test using:<br>
<code>adb install -r android-cts/repository/testcases/CtsDeviceAdmin.apk</code><br>
-On the device, enable only the two
+<p>
+In Settings > Security > Select device administrators, enable the two
<code>android.deviceadmin.cts.CtsDeviceAdminReceiver*</code> device
-administrators under: <strong>Settings > Security > Select device
-administrators</strong>. Make sure the
+administrators. Ensure the
<code>android.deviceadmin.cts.CtsDeviceAdminDeactivatedReceiver</code> and any
-other preloaded device administrators stay disabled in the same menu.</p>
+other preloaded device administrators remain disabled.
+</p>
<li>Copy the CTS media files to the device as follows:
-<p class="note"><strong>Note:</strong> For CTS 2.3 R12 and beyond, if the
+<p class="note"><strong>Note:</strong> For CTS 2.3 R12 and later, if the
device supports video codecs, the CTS media files must be copied to the
device.</p>
<ul>
