| page.title=Interpreting CTS results |
| @jd:body |
| |
| <!-- |
| Copyright 2015 The Android Open Source Project |
| |
| Licensed under the Apache License, Version 2.0 (the "License"); |
| you may not use this file except in compliance with the License. |
| You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| |
| <p>The CTS test results are placed in the file:</p> |
| <pre> |
| $CTS_ROOT/android-cts/repository/results/<start_time>.zip |
| </pre> |
| |
| <p>If you have built the CTS yourself, <em>$CTS_ROOT</em> will resemble the |
| path <em>out/host/linux-x86/cts</em> but differ by platform. This reflects the |
| path where you have uncompressed the prebuilt official CTS <a |
| href="downloads.html">downloaded</a> from this site.</p> |
| |
| <p>Inside the zip, the testResult.xml file contains the actual results—open |
| this file in any web browser (HTML5 compatible browser recommended) to view the |
| test results. It will resemble the following screenshots.</p> |
| |
| <p class="note"><strong>Note:</strong> The results are provided to help you |
| ensure the software remains compatible throughout the development process and |
| act as a common format for communicating the compatibility status of your |
| device with other parties.</p> |
| |
| <img src="images/cts-test-summary.png" alt="CTS test summary" id="figure1" /> |
| <p class="img-caption"> |
| <strong>Figure 1.</strong> CTS test summary |
| </p> |
| |
| <p>If testResult.xml displays a blank page when using the Chrome browser, |
| <a href="https://www.chromium.org/developers/how-tos/run-chromium-with-flags">change |
| your browser configuration</a> to enable the |
| <em>--allow-file-access-from-files</em> command line flag.</p> |
| |
| <p>The Device Information section provides details about the device, firmware |
| (make, model, firmware build, platform), and device hardware (screen |
| resolution, keypad, screen type). To access device information, click the link |
| above Test Summary.</p> |
| |
| <p>The <em>Test Summary</em> section provides executed test plan details, like |
| the CTS plan name and execution start and end times. It also presents an |
| aggregate summary of the number of tests that passed, failed, timed out, or |
| could not be executed.</p> |
| <p>The next section also provides a summary of tests passed per package.</p> |
| |
| <img src="images/cts-test-report.png" alt="CTS detailed test report" id="figure2" /> |
| <p class="img-caption"> |
| <strong>Figure 2.</strong> CTS test report |
| </p> |
| |
| <p>This is followed by details of the the actual tests that were executed. The |
| report lists the test package, test suite, test case, and the executed tests. |
| It shows the result of the test execution—pass, fail, timed out, or not |
| executed. In the event of a test failure details are provided to help diagnose |
| the cause.</p> |
| |
| <p>Further, the stack trace of the failure is available in the XML file but is |
| not included in the report to ensure brevity—viewing the XML file with a text |
| editor should provide details of the test failure (search for the |
| <em><Test></em> tag corresponding to the failed test and look within it |
| for the <em><StackTrace></em> tag).</p> |