en/compatibility/cts/verifier.html - platform/docs/source.android.com - Git at Google

 <html devsite>
   <head>
     <title>Using CTS Verifier</title>
     <meta name="project_path" value="/_project.yaml" />
     <meta name="book_path" value="/_book.yaml" />
   </head>
   <body>
   <!--
       Copyright 2017 The Android Open Source Project

       Licensed under the Apache License, Version 2.0 (the "License");
       you may not use this file except in compliance with the License.
       You may obtain a copy of the License at

           http://www.apache.org/licenses/LICENSE-2.0

       Unless required by applicable law or agreed to in writing, software
       distributed under the License is distributed on an "AS IS" BASIS,
       WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
       See the License for the specific language governing permissions and
       limitations under the License.
   -->


 <p>The Android Compatibility Test Suite Verifier (CTS Verifier) supplements the
 Compatibility Test Suite (CTS). While CTS checks APIs and functions that can be
 automated, CTS Verifier provides tests for APIs and functions that cannot be
 tested on a stationary device without manual input, such as audio quality,
 touchscreen, accelerometer, camera, etc.</p>

 <h2 id=test_preparation>Requirements</h2>
 <p>Before running CTS Verifier, ensure you have the following equipment:</p>
 <ul>
 <li>Android device that has verified Android API compatibility by successfully
 passing the CTS. This will be the device-under-test (DUT).</li>
 <li>Linux computer with USB 2.0 compatible port. All connections to the DUT will
 be through this port.</li>
 <li>Second Android device with a known compatible Bluetooth, Wi-Fi direct, and
 NFC Host Card Emulation (HCE) implementation.</li>
 <li>A Wi-Fi router configured with access point name and password. The router
 should have the ability to disconnect from the internet, but not powered off.</li>
 </ul>

 <h2 id=setup>Setting up</h2>
 <p>To setup the CTS Verifier testing environment:</p>
 <ol>
 <li>On the Linux computer:
 <ul>
 <li>Install the <a href="http://developer.android.com/sdk/index.html">Android
 SDK</a>.</li>
 <li>Download the
 <a href="/compatibility/cts/downloads.html">CTS Verifier APK</a> for the
 version of Android to test.</li>
 </ul>
 </li>
 <li>Connect the DUT to the Linux computer.
 <li>From a terminal on the Linux computer, install <code>CtsVerifier.apk</code>
 on the DUT.
 <pre class="devsite-terminal devsite-click-to-copy">
 adb install -r -g CtsVerifier.apk
 </pre>
 </li>
 <li>Ensure the DUT has the system data and time set correctly.</li>
 </ol>

 <h2 id=cts_test_procedure>Running</h2>
 <p>Launch the CTS Verifier application by tapping the CTS icon on the DUT:</p>
 <img src="/compatibility/cts/images/cts-verifier-icon.png" alt="CTS Verifier
 icon in launcher" id="figure1" />
 <figcaption><strong>Figure 1.</strong> CTS Verifier icon</figcaption>

 <p>The app displays several test sets available for manual verification:</p>
 <img src="/compatibility/cts/images/cts-verifier-menu.png" alt="CTS Verifier
 menu of tests" id="figure2" />
 <figcaption><strong>Figure 2.</strong> CTS Verifier menu of tests.</figcaption>

 <p>Each test contains a set of common elements (Info, Pass, Fail):</p>
 <img src="/compatibility/cts/images/video-verifier.png" alt="Streaming video
 quality verifier" id="figure3" />
 <figcaption><strong>Figure 3.</strong> Test elements.</figcaption>

 <ul>
 <li><strong>Info</strong> (?). Tap to display test instructions. Also appears
 automatically the first time a test is opened.</li>
 <li><strong>Pass</strong> (✓). Tap if the DUT meets the test requirements per
 the Info instructions.</li>
 <li><strong>Fail</strong> (!). Tap if the DUT does not meet the test
 requirements per the Info instructions.</li>
 </ul>

 <aside class="note"><strong>Note:</strong> In some tests, Pass/Fail is
 determined automatically.</aside>

 <p>Some tests, such as the USB accessory mode and camera calibration test,
 require additional test setup and instructions as detailed in the following
 sections.</p>

 <h3 id=usb_accessory>Testing USB accessory mode</h3>
 <p>The USB Accessory test requires a Linux computer to run the USB desktop
 machine (host) program.</p>
 <ol>
 <li>Connect the DUT to the Linux computer.</li>
 <li>On the computer, execute the <code>cts-usb-accessory</code> program from the
 CTS Verifier package:
 <pre class="devsite-terminal devsite-click-to-copy">./cts-usb-accessory</pre>
 </li>
 <li>Wait for a popup message to appear on the DUT, then select
 <strong>OK</strong>.<br>
 <img src="/compatibility/cts/images/screen-lock-test.png" alt="CTS Verifier
 usb accessory test" id="figure4" />
 <figcaption><strong>Figure 4.</strong> USB accessory test.</figcaption>
 </li>
 <li>Go to the USB Accessory Test in the CTS Verifier application.</li>
 <li>On the computer, review the output from the console. Example output:
 <pre class="devsite-click-to-copy">
 CTS USB Accessory Tester
 Found possible Android device (413c:2106) - attempting to switch to accessory
 mode...
 Failed to read protocol version
 Found Android device in accessory mode (18d1:2d01)...
 [RECV] Message from Android device #0
 [SENT] Message from Android accessory #0
 [RECV] Message from Android device #1
 [SENT] Message from Android accessory #1
 [RECV] Message from Android device #2
 [SENT] Message from Android accessory #2
 [RECV] Message from Android device #3
 [SENT] Message from Android accessory #3
 [RECV] Message from Android device #4
 [SENT] Message from Android accessory #4
 [RECV] Message from Android device #5
 [SENT] Message from Android accessory #5
 [RECV] Message from Android device #6
 [SENT] Message from Android accessory #6
 [RECV] Message from Android device #7
 [SENT] Message from Android accessory #7
 [RECV] Message from Android device #8
 [SENT] Message from Android accessory #8
 [RECV] Message from Android device #9
 [SENT] Message from Android accessory #9
 [RECV] Message from Android device #10
 [SENT] Message from Android accessory #10
 </pre>
 </li>
 </ol>

 <h3 id=camera_field_of_view_calibration>Calibrating camera field of view</h3>
 <p>Use the field of view calibration procedure to quickly determine the device
 field of view with moderate accuracy.</p>

 <ol>
 <li>Setup the test environment:
 <ol>
 <li>Print the
 <a href="/compatibility/calibration-pattern.pdf">calibration-pattern.pdf</a>
 target file on 11” x 17” or A3 size paper.</li>
 <li>Mount the printed pattern on a rigid backing.</li>
 <li>Orient the camera device and the printed target as shown below:<br>
 <img src="/compatibility/cts/images/camera-printed-target.png" alt="Camera
 printed target" id="figure5" />
 <figcaption><strong>Figure 5.</strong> Camera printed target</figcaption>
 </li>
 </ol>
 </li>
 <li>Set the target width:
 <ol>
 <li>Measure the distance (in centimeters) between the solid lines on the target
 pattern to account for printing inaccuracies (~38 cm).</li>
 <li>Start the calibration application.</li>
 <li>Press the setup button and select <em>Marker distance</em>.</li>
 <li>Measure and enter the distance to the target pattern (~100 cm).</li>
 <li>Press the back button to return to the calibration preview.</li>
 </ol>
 </li>
 <li>Verify the device and target are placed as shown in the figure and the
 correct distances have been entered into the setup dialog. The preview will
 display the image with a vertical line overlaid onto it; this line should align
 with the center line of the target pattern. The transparent grid can be used
 with the other vertical lines to ensure that the optical axis is orthogonal to
 the target.</li>
 <li>Run the calibration test:
 <ol>
 <li>Select an image resolution (using selector at the bottom left), then tap the
 screen to take a photo. The test enters calibration mode and displays the photo
 with two vertical lines overlaid onto the image.</li>
 <li>Determine accuracy:
 <ul>
 <li>If the lines align with the vertical lines on the target pattern within a
 few pixels, the reported field of view for the selected resoultion is accurate.
 </li>
 <li>If the lines do not align, the reported field of view is inaccurate. To
 correct, adjust the slider at the bottom of the screen until the overlay aligns
 with the target pattern as closely as possible. When the overlay and the target
 pattern image are aligned, the displayed field of view will be a close
 approximation to the correct value. The reported field of view should be
 within +/-1 degree of the calibration value.</li>
 </ul>
 </li>
 <li>Press back button and repeat the calibration test for all image resolutions
 supported by the DUT.</li>
 </ol>
 </li>
 </ol>

 <h2 id=exporting_test_reports>Exporting results</h2>
 <p>After all tests complete, you can save the results as a report and download
 to a computer. Report names are automatically time-stamped based on the DUT
 system time.</p>

 <ol>
 <li>Tap the <strong>Save (disk)</strong> icon.<br>
 <img src="/compatibility/cts/images/verifier-save-icon.png" alt="CTS Verifier
 Save icon" id="figure6" />
 <figcaption><strong>Figure 6.</strong> CTS Verifier Save icon.</figcaption>
 <aside class="note"><strong>Note:</strong> Android 7.0 and higher do not include
 the preview feature.</aside>
 </li>
 <li>Wait for the popup message to display the path to the saved report (e.g.
 <code>/sdcard/verifierReports/ctsVerifierReport-date-time.zip</code>), then
 record the path.<br>
 <img src="images/path-saved-report.png" alt="CTS Verifier path to saved report"
 id="figure7" />
 <figcaption><strong>Figure 7.</strong> CTS Verifier path to saved
 report.</figcaption>
 </li>
 <li>Connect the DUT to the Linux computer.</li>
 <li>From the Android SDK installation on the Linux computer, download reports
 from the connected device using <code>adb pull CTSVerifierReportPath</code>.
 <ul>
 <li>For Android 7.x and later, download all reports using:
 <pre class="devsite-terminal devsite-click-to-copy">
 adb pull /sdcard/verifierReports
 </pre>
 </li>
 <li>For Android 6.0 and earlier, download all reports using:
 <pre class="devsite-terminal devsite-click-to-copy">
 adb pull /mnt/sdcard/ctsVerifierReports/
 </pre>
 </li>
 </ul>
 </li>
 <li>To clear Pass/Fail results, select the results in the CTS Verifier app and
 select <em>Menu > Clear</em>.</li>
 </ol>

   </body>
 </html>
	<html devsite>
	<head>
	<title>Using CTS Verifier</title>
	<meta name="project_path" value="/_project.yaml" />
	<meta name="book_path" value="/_book.yaml" />
	</head>
	<body>
	<!--
	Copyright 2017 The Android Open Source Project

	Licensed under the Apache License, Version 2.0 (the "License");
	you may not use this file except in compliance with the License.
	You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

	Unless required by applicable law or agreed to in writing, software
	distributed under the License is distributed on an "AS IS" BASIS,
	WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
	See the License for the specific language governing permissions and
	limitations under the License.
	-->



	<p>The Android Compatibility Test Suite Verifier (CTS Verifier) supplements the
	Compatibility Test Suite (CTS). While CTS checks APIs and functions that can be
	automated, CTS Verifier provides tests for APIs and functions that cannot be
	tested on a stationary device without manual input, such as audio quality,
	touchscreen, accelerometer, camera, etc.</p>

	<h2 id=test_preparation>Requirements</h2>
	<p>Before running CTS Verifier, ensure you have the following equipment:</p>
	<ul>
	<li>Android device that has verified Android API compatibility by successfully
	passing the CTS. This will be the device-under-test (DUT).</li>
	<li>Linux computer with USB 2.0 compatible port. All connections to the DUT will
	be through this port.</li>
	<li>Second Android device with a known compatible Bluetooth, Wi-Fi direct, and
	NFC Host Card Emulation (HCE) implementation.</li>
	<li>A Wi-Fi router configured with access point name and password. The router
	should have the ability to disconnect from the internet, but not powered off.</li>
	</ul>

	<h2 id=setup>Setting up</h2>
	<p>To setup the CTS Verifier testing environment:</p>
	<ol>
	<li>On the Linux computer:
	<ul>
	<li>Install the <a href="http://developer.android.com/sdk/index.html">Android
	SDK</a>.</li>
	<li>Download the
	<a href="/compatibility/cts/downloads.html">CTS Verifier APK</a> for the
	version of Android to test.</li>
	</ul>
	</li>
	<li>Connect the DUT to the Linux computer.
	<li>From a terminal on the Linux computer, install <code>CtsVerifier.apk</code>
	on the DUT.
	<pre class="devsite-terminal devsite-click-to-copy">
	adb install -r -g CtsVerifier.apk
	</pre>
	</li>
	<li>Ensure the DUT has the system data and time set correctly.</li>
	</ol>

	<h2 id=cts_test_procedure>Running</h2>
	<p>Launch the CTS Verifier application by tapping the CTS icon on the DUT:</p>
	<img src="/compatibility/cts/images/cts-verifier-icon.png" alt="CTS Verifier
	icon in launcher" id="figure1" />
	<figcaption><strong>Figure 1.</strong> CTS Verifier icon</figcaption>

	<p>The app displays several test sets available for manual verification:</p>
	<img src="/compatibility/cts/images/cts-verifier-menu.png" alt="CTS Verifier
	menu of tests" id="figure2" />
	<figcaption><strong>Figure 2.</strong> CTS Verifier menu of tests.</figcaption>

	<p>Each test contains a set of common elements (Info, Pass, Fail):</p>
	<img src="/compatibility/cts/images/video-verifier.png" alt="Streaming video
	quality verifier" id="figure3" />
	<figcaption><strong>Figure 3.</strong> Test elements.</figcaption>

	<ul>
	<li><strong>Info</strong> (?). Tap to display test instructions. Also appears
	automatically the first time a test is opened.</li>
	<li><strong>Pass</strong> (✓). Tap if the DUT meets the test requirements per
	the Info instructions.</li>
	<li><strong>Fail</strong> (!). Tap if the DUT does not meet the test
	requirements per the Info instructions.</li>
	</ul>

	<aside class="note"><strong>Note:</strong> In some tests, Pass/Fail is
	determined automatically.</aside>

	<p>Some tests, such as the USB accessory mode and camera calibration test,
	require additional test setup and instructions as detailed in the following
	sections.</p>

	<h3 id=usb_accessory>Testing USB accessory mode</h3>
	<p>The USB Accessory test requires a Linux computer to run the USB desktop
	machine (host) program.</p>
	<ol>
	<li>Connect the DUT to the Linux computer.</li>
	<li>On the computer, execute the <code>cts-usb-accessory</code> program from the
	CTS Verifier package:
	<pre class="devsite-terminal devsite-click-to-copy">./cts-usb-accessory</pre>
	</li>
	<li>Wait for a popup message to appear on the DUT, then select
	<strong>OK</strong>.<br>
	<img src="/compatibility/cts/images/screen-lock-test.png" alt="CTS Verifier
	usb accessory test" id="figure4" />
	<figcaption><strong>Figure 4.</strong> USB accessory test.</figcaption>
	</li>
	<li>Go to the USB Accessory Test in the CTS Verifier application.</li>
	<li>On the computer, review the output from the console. Example output:
	<pre class="devsite-click-to-copy">
	CTS USB Accessory Tester
	Found possible Android device (413c:2106) - attempting to switch to accessory
	mode...
	Failed to read protocol version
	Found Android device in accessory mode (18d1:2d01)...
	[RECV] Message from Android device #0
	[SENT] Message from Android accessory #0
	[RECV] Message from Android device #1
	[SENT] Message from Android accessory #1
	[RECV] Message from Android device #2
	[SENT] Message from Android accessory #2
	[RECV] Message from Android device #3
	[SENT] Message from Android accessory #3
	[RECV] Message from Android device #4
	[SENT] Message from Android accessory #4
	[RECV] Message from Android device #5
	[SENT] Message from Android accessory #5
	[RECV] Message from Android device #6
	[SENT] Message from Android accessory #6
	[RECV] Message from Android device #7
	[SENT] Message from Android accessory #7
	[RECV] Message from Android device #8
	[SENT] Message from Android accessory #8
	[RECV] Message from Android device #9
	[SENT] Message from Android accessory #9
	[RECV] Message from Android device #10
	[SENT] Message from Android accessory #10
	</pre>
	</li>
	</ol>

	<h3 id=camera_field_of_view_calibration>Calibrating camera field of view</h3>
	<p>Use the field of view calibration procedure to quickly determine the device
	field of view with moderate accuracy.</p>

	<ol>
	<li>Setup the test environment:
	<ol>
	<li>Print the
	<a href="/compatibility/calibration-pattern.pdf">calibration-pattern.pdf</a>
	target file on 11” x 17” or A3 size paper.</li>
	<li>Mount the printed pattern on a rigid backing.</li>
	<li>Orient the camera device and the printed target as shown below:<br>
	<img src="/compatibility/cts/images/camera-printed-target.png" alt="Camera
	printed target" id="figure5" />
	<figcaption><strong>Figure 5.</strong> Camera printed target</figcaption>
	</li>
	</ol>
	</li>
	<li>Set the target width:
	<ol>
	<li>Measure the distance (in centimeters) between the solid lines on the target
	pattern to account for printing inaccuracies (~38 cm).</li>
	<li>Start the calibration application.</li>
	<li>Press the setup button and select <em>Marker distance</em>.</li>
	<li>Measure and enter the distance to the target pattern (~100 cm).</li>
	<li>Press the back button to return to the calibration preview.</li>
	</ol>
	</li>
	<li>Verify the device and target are placed as shown in the figure and the
	correct distances have been entered into the setup dialog. The preview will
	display the image with a vertical line overlaid onto it; this line should align
	with the center line of the target pattern. The transparent grid can be used
	with the other vertical lines to ensure that the optical axis is orthogonal to
	the target.</li>
	<li>Run the calibration test:
	<ol>
	<li>Select an image resolution (using selector at the bottom left), then tap the
	screen to take a photo. The test enters calibration mode and displays the photo
	with two vertical lines overlaid onto the image.</li>
	<li>Determine accuracy:
	<ul>
	<li>If the lines align with the vertical lines on the target pattern within a
	few pixels, the reported field of view for the selected resoultion is accurate.
	</li>
	<li>If the lines do not align, the reported field of view is inaccurate. To
	correct, adjust the slider at the bottom of the screen until the overlay aligns
	with the target pattern as closely as possible. When the overlay and the target
	pattern image are aligned, the displayed field of view will be a close
	approximation to the correct value. The reported field of view should be
	within +/-1 degree of the calibration value.</li>
	</ul>
	</li>
	<li>Press back button and repeat the calibration test for all image resolutions
	supported by the DUT.</li>
	</ol>
	</li>
	</ol>

	<h2 id=exporting_test_reports>Exporting results</h2>
	<p>After all tests complete, you can save the results as a report and download
	to a computer. Report names are automatically time-stamped based on the DUT
	system time.</p>

	<ol>
	<li>Tap the <strong>Save (disk)</strong> icon.<br>
	<img src="/compatibility/cts/images/verifier-save-icon.png" alt="CTS Verifier
	Save icon" id="figure6" />
	<figcaption><strong>Figure 6.</strong> CTS Verifier Save icon.</figcaption>
	<aside class="note"><strong>Note:</strong> Android 7.0 and higher do not include
	the preview feature.</aside>
	</li>
	<li>Wait for the popup message to display the path to the saved report (e.g.
	<code>/sdcard/verifierReports/ctsVerifierReport-date-time.zip</code>), then
	record the path.<br>
	<img src="images/path-saved-report.png" alt="CTS Verifier path to saved report"
	id="figure7" />
	<figcaption><strong>Figure 7.</strong> CTS Verifier path to saved
	report.</figcaption>
	</li>
	<li>Connect the DUT to the Linux computer.</li>
	<li>From the Android SDK installation on the Linux computer, download reports
	from the connected device using <code>adb pull CTSVerifierReportPath</code>.
	<ul>
	<li>For Android 7.x and later, download all reports using:
	<pre class="devsite-terminal devsite-click-to-copy">
	adb pull /sdcard/verifierReports
	</pre>
	</li>
	<li>For Android 6.0 and earlier, download all reports using:
	<pre class="devsite-terminal devsite-click-to-copy">
	adb pull /mnt/sdcard/ctsVerifierReports/
	</pre>
	</li>
	</ul>
	</li>
	<li>To clear Pass/Fail results, select the results in the CTS Verifier app and
	select <em>Menu > Clear</em>.</li>
	</ol>

	</body>
	</html>