en/compatibility/cts/camera-hal.html - platform/docs/source.android.com - Git at Google

 <html devsite>
   <head>
     <title>Camera HAL Testing Checklist</title>
     <meta name="project_path" value="/_project.yaml" />
     <meta name="book_path" value="/_book.yaml" />
   </head>
   <body>
   <!--
       Copyright 2017 The Android Open Source Project

       Licensed under the Apache License, Version 2.0 (the "License");
       you may not use this file except in compliance with the License.
       You may obtain a copy of the License at

           http://www.apache.org/licenses/LICENSE-2.0

       Unless required by applicable law or agreed to in writing, software
       distributed under the License is distributed on an "AS IS" BASIS,
       WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
       See the License for the specific language governing permissions and
       limitations under the License.
   -->

 {% include "_versions.html" %}

 <p>This document lists all tests available for evaluating the Android camera
 hardware abstraction layer (HAL). It is intended for original equipment
 manufacturers (OEMs) and application
 processor (AP) vendors so they may ensure proper implementation of the camera
 HAL with minimum defects. Although this is a voluntary addition to the Android
 Compatibility Test Suite (CTS), it greatly increases camera test coverage and
 will certainly identify potential bugs.</p>

 <p>By passing these tests, OEMs validate
 whether they have properly integrated the Android
 camera hardware
 abstraction layer (HAL) 3 interfaces. When conforming with all items in the
 checklist, a device implementation may be considered <em>full</em> with respect
 to the Android Camera HAL interfaces. This will in turn enable a device to
 properly support the
 <a href="https://developer.android.com/reference/android/hardware/camera2/package-summary" class="external">
   <code>android.hardware.camera2</code></a> package that
 camera apps build upon.</p>

 <h2 id=camera_hal_3_2_specification>Camera HAL3 specification</h2>

 <p>The <a href="/devices/camera/camera3">Android Camera HAL3</a> specification
 is the authoritative source of
 information on what devices must satisfy; this page provides a summary
 of all tests that can be used as a checklist. Camera HAL implementers (e.g. AP
 vendors) should go through the Camera HAL3 specification line-by-line and ensure
 their devices conform to it.</p>

 <p>The current HAL specification is defined in these files within the Android
 5.0 and later generic Android Platform Development Kit (PDK):</p>

 <ul>
   <li><em>Camera HAL 3.x interface and spec</em>: <code><a
 href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/camera3.h"
 class="external">hardware/libhardware/include/hardware/camera3.h</a></code>,
 <code><a
 href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/camera_common.h"
 class="external">hardware/libhardware/include/hardware/camera_common.h</a></code>
   <li><em>Camera HAL 3.x metadata spec</em>: <code><a
 href="https://android.googlesource.com/platform/system/media/+/master/camera/docs/docs.html"
 class="external">system/media/camera/docs/docs.html</a></code>
   <li><em>HAL pixel format interface and spec</em>: <code><a
 href="https://android.googlesource.com/platform/system/core/+/master/libsystem/include/system/graphics.h"
 class="external">system/core/libsystem/include/system/graphics.h</a></code>
 </ul>

 <h2 id=camera_test_types>Camera test types</h2>

 <p>Here are the primary types of tests available for the latest Android camera
 along with references to associated instructions below:</p>

 <ul>
   <li><em><a href="#vendor_test_suite">Vendor Test Suite (VTS)</a>:</em> Tests that directly test the
   camera HAL interface
   <li><em><a href="#cts_tests">Compatibility Test Suite (CTS)</a></em>:
   Standard, automated Android
   tests to ensure device compatibility - see the
   <a href="/compatibility/cts/index.html">CTS introduction</a> and the <a
   href="/devices/tech/test_infra/tradefed/index.html">Trade Federation
   Overview</a>
   <li><em><a href="#its_tests">Image Test Suite (ITS)</a>:</em> Manually run
   tests to ensure image correctness - see the top-level and test-specific
   <code>README</code> files and <code>tutorial.py</code> for setup instructions
   <li><em><a
   href="#manual_testingcam_tests">Manual TestingCam tests</a>:</em>
   Run from the source in <code>pdk/apps/TestingCamera/</code>
   <li><em><a href="#manual_testingcam2_tests">Manual TestingCam2.1
   tests</a>:</em> Run from the source in <code>pdk/apps/TestingCamera2/</code>
 </ul>

 <p>All of these test types are described in detail below. These tests are
 presented in the chronological order in which OEMs are expected to execute
 them.</p>

 <p>For instance, if a device fails the native tests, it will assuredly fail the
 subsequent Compatibility Test Suite (CTS) tests. And if a device fails CTS,
 there is little use in proceeding to the Image Test Suite (ITS). We recommend
 addressing failures in each test type before proceeding to the next set of
 tests.</p>

 <h2 id=vendor_test_suite>Vendor Test Suite (VTS) tests</h2>

 <p>The Android Vendor Test Suite (VTS) is a testing suite that works on the
 HIDL interface level. For more information on using VTS, see
 <a href="/compatibility/vts/">Vendor Test Suite</a>.</p>

 <h2 id=cts_tests>Compatibility Test Suite (CTS) tests</h2>

 <p>Camera Android Compatibility Test Suite (CTS) tests focus upon device
 compatibility. For more information on the recommended test environment for camera CTS, see <a href="/compatibility/cts/setup#camera">Set up CTS</a>.</p>

 <p>The starting path for Camera CTS tests is: <code>platform/cts</code>.</p>

 <p>When running Camera CTS for devices that support external cameras (such as
 USB webcams), you must have a device plugged in when running CTS or the tests
 will automatically fail. Examples of external cameras include: <a
 href="https://www.logitech.com/en-us/product/hd-pro-webcam-c920"
 class="external">Logitech HD Pro Webcam C920</a> and the <a
 href="https://www.microsoft.com/accessories/en-us/products/webcams/lifecam-hd-3000/t3h-00011"
 class="external">Microsoft LifeCam HD-3000</a>.</p>

 <p>See the <a href="/compatibility/cts/index.html">CTS
 introduction</a> and its subpages for general instructions on running CTS.</p>

 <h3 id=cts_tests_for_the_android_hardware_camera_api>CTS tests for
 the <code>android.hardware.Camera</code> API</h3>

 <p>Find these camera tests under <code>cts/tests/tests/</code>:</p>

 <ul>
   <li><code>hardware/src/android/hardware/cts/CameraTest.java</code>
   <li><code>hardware/src/android/hardware/cts/CameraGLTest.java</code>
   <li><code>hardware/src/android/hardware/cts/Camera_SizeTest.java</code>
   <li><code>permission/src/android/permission/cts/CameraPermissionTest.java</code>
 </ul>

 <h3 id=cts_tests_for_the_android_hardware_camera2_api>CTS tests for
 the <code>android.hardware.camera2</code> API</h3>

 <p>Find these camera tests under <code>cts/tests/tests/</code>:</p>

 <ul>
   <li><code>hardware/src/android/hardware/camera2/cts/*</code>
   <li><code>permission/src/android/permission/cts/Camera2PermissionTest.java</code>
 </ul>

 <h3 id=cts_verifier_camera_tests>CTS Verifier camera tests</h3>

 <p>Find these camera tests under:
 <code>cts/apps/CtsVerifier/src/com/android/cts/verifier/camera/*</code></p>

 <h2 id=its_tests>Image Test Suite (ITS) tests</h2>

 <aside class="Note"><strong>Note:</strong>
 <a href="/devices/camera/camera3">Camera HAL3</a> is strongly recommended for
   devices running Android {{ androidPVersionNumber }} or higher as Camera HAL1
   has been deprecated. This
   also allows for a smooth phase out of the deprecated
   <a href="https://developer.android.com/reference/android/hardware/Camera" class="external">
     Camera API</a>.</aside>

 <p>The Camera Image Test Suite (ITS) tests focus on image correctness.
   To perform the tests, run the Python scripts on a workstation with the
   Android device connected over USB.</p>

 <p>The Camera ITS infrastructure and tests are located in the
 <a href="https://android.googlesource.com/platform/cts/+/master/apps/CameraITS"
 class="external">
   <code>cts/apps/CameraITS</code></a> directory. Each test resides in a
   <code>tests/scene<var>#</var></code> subdirectory.</p>

 To set up the test environment, run:

 <pre class="devsite-click-to-copy">
 <code class="devsite-terminal">extract root/out/host/linux-x86/cts-verfier/android-cts-verifier.zip</code>
 <code class="devsite-terminal">cd android-cts-verifier</code>
 <code class="devsite-terminal">adb install -r CtsVerifier.apk</code>
 <code class="devsite-terminal">cd CameraITS</code>
 <code class="devsite-terminal">source build/envsetup.sh</code>
 </pre>

 <aside class="note"><strong>Note:</strong> Because ITS is a CTS Verifier subtest,
 start CTS Verifier and the ITS subtest before running the Python scripts so they
 have processes with which to communicate.</aside>

 <p>For more information on how to set up and run the tests, see the
 <code>CameraITS</code> PDF file in <code>cts/apps/CameraITS</code>. See
 <a href="https://android.googlesource.com/platform/cts/+/master/apps/CameraITS/tests/tutorial.py"
    class="external">
   <code>tutorial.py</code></a> in the <code>tests</code> subdirectory for a
 guide on how to use the scripts.</p>

 <p>The ITS static tests (scenes 0-5) can run with any operating
 system with the required Python 2.7 environment. However, the
 <code>sensor_fusion</code>
 test with the <a href="/compatibility/cts/sensor-fusion-quick-start">Sensor
 Fusion Box</a> must be run with the
 Linux operating system.
 </p>

 <p>The recommended setup for scenes 0-4
 is described in <a
 href="/compatibility/cts/camera-its-box">Camera ITS-in-a-box</a>. The
 recommended setup for the sensor_fusion scene is described in <a
 href="/compatibility/cts/sensor-fusion-quick-start">Sensor Fusion Box Quick
   Start Guide</a>.</p>

 <p>To run ITS manually, prepare a simple physical environment with a
 specific, reusable target
 such as a white wall, a grey card, and a desk lamp. Mount the Android device on
   a tripod and run the scripts to test the camera functions. Most tests
 are pass or fail but some offer metrics.</p>

 <p>These scripts test scenarios that are not tested in CTS and are an important
   component of the HAL 3.2 test plan.</p>

 <p>ITS tests either pass or fail. All mandated
 tests in each scene folder must pass. Tests that are not mandated can
 fail and still count as a pass in
 <code>CtsVerifier</code>.</p>

 <h3 id="scene0_to_scene4_tests">Scene 0 to scene 4 tests</h3>

 <p>These scenes represent a large part of ITS testing and are included as PDF
 files in the <code>scene</code> folder. To automate these tests, use the <a
 href="/compatibility/cts/camera-its-box">Camera ITS-in-a-box</a> system.</p>

 <ul>
   <li>Scene 0: Requires no setup.</li>
   <li>Scene 1: Requires a grey card.</li>
   <li>Scene 2: Requires a face scene.</li>
   <li>Scene 3: ISO12233 chart.</li>
   <li>Scene 4: Custom scene with a circle inside a square.</li>
 </ul>

 <h3 id="scene5_tests">Scene 5 tests</h3>
 <p>Scene 5 tests require a diffuser to be placed on top of the camera.</p>

 <h3 id="sensor_fusion_tests">Sensor fusion tests</h3>

 <p>Sensor fusion tests require specific camera motion to test the timestamp
 difference between the camera and the gyroscope for AR and VR applications. This
 test is skipped if no gyroscope is included or if the <code>REALTIME</code>
 parameter is not enabled. The <code>sensor_fusion</code> test can be automated
 with the <a
 href="/compatibility/cts/sensor-fusion-quick-start">Sensor Fusion Box</a>.</p>

 <h2 id=media_framework_tests>Media Framework tests</h2>

 <p>Pass all of the camera-related media tests in MediaFrameworkTest. Please
 note, these tests require the mediaframeworktest.apk be installed on the Android
 device. You will need to <code>make mediaframeworktest</code> and then use adb
 to install the resulting .apk. Example commands are included below.</p>

 <p>The starting path for Camera-related media framework tests is:
 <code>platform/frameworks/base</code></p>

 <p>Find the source code for the tests here:
 <code>frameworks/base/media/tests/MediaFrameworkTest</code></p>

 <p>To set up these tests:</p>

 <pre class="devsite-click-to-copy">
 <code class="devsite-terminal">make mediaframeworktest</code>
 <code class="devsite-terminal">adb install out/target/product/<var>name</var>/data/app/mediaframeworktest.apk</code>
 </pre>

     <p>Where the <var>name</var> variable represents the
   directory containing the vendor's product.</p>

 <p>Find all of the tests in the following directory or its subdirectories:</p>

 <pre class="devsite-click-to-copy">
 frameworks/base/media/tests/MediaFrameworkTest/src/com/android/mediaframeworktest
 </pre>

 <p>Each subdirectory represents a class of tests:</p>

 <ul>
   <li><code>functional/</code>
   <li><code>integration/</code>
   <li><code>performance/</code>
   <li><code>power/</code>
   <li><code>stress/</code>
   <li><code>unit/</code>
 </ul>

 <h3 id=running_media_framework_tests>Running Media Framework tests</h3>

 <p>To see all of the available tests::</p>

 <pre class="devsite-terminal devsite-click-to-copy">
 adb shell pm list instrumentation
 </pre>

 <p>This will yield results resembling:</p>

 <pre class="devsite-click-to-copy">
 instrumentation:com.android.mediaframeworktest/.MediaFrameworkIntegrationTestRunner
 (target=com.android.mediaframeworktest)
 instrumentation:com.android.mediaframeworktest/.MediaRecorderStressTestRunner
 (target=com.android.mediaframeworktest)
 instrumentation:com.android.mediaframeworktest/.MediaFrameworkPerfTestRunner
 (target=com.android.mediaframeworktest)
 instrumentation:com.android.mediaframeworktest/.MediaFrameworkPowerTestRunner
 (target=com.android.mediaframeworktest)
 </pre>

 <p>Identify and extract the component (between <code>instrumentation:</code>
 and <code>(target=com.android.mediaframeworktest) </code>from each test line.
 The component is composed of the target package name
 (<code>com.android.mediaframeworktest</code>) and the test runner name
 (<code>MediaFramework<type>TestRunner</code>).</p>

 <p>For instance:</p>

 <pre class="devsite-click-to-copy">
 com.android.mediaframeworktest/.MediaFrameworkIntegrationTestRunner
 com.android.mediaframeworktest/.MediaRecorderStressTestRunner
 com.android.mediaframeworktest/.MediaFrameworkPerfTestRunner
 com.android.mediaframeworktest/.MediaFrameworkPowerTestRunner
 </pre>

 <p>You may then pass each component to <code>adb shell am instrument</code> like
 so:</p>

 <pre class="devsite-terminal devsite-click-to-copy">
 adb shell am instrument -w <var>component.name</var>
 </pre>

     <p>Where <code><var>component.name</var></code> equals the extracted value
       above. For
 example:</p>

 <pre class="devsite-terminal devsite-click-to-copy">
 adb shell am instrument -w com.android.mediaframeworktest/.MediaFrameworkIntegrationTestRunner
 </pre>

 <p>Please note, while the class path is the Java package + class name, the
 instrumentation package isn't necessarily the same as the Java package. Make
 sure you use the AndroidManifest.xml package when concatenating the component
 name, not the Java package in which the test runner class resides.</p>

 <p>To run a single class of tests, pass the -e class <test-class> argument, like
 so:</p>

 <pre class="devsite-terminal devsite-click-to-copy">
 adb shell am instrument -e class com.android.mediaframeworktest.integration.CameraBinderTest -w com.android.mediaframeworktest/.MediaFrameworkIntegrationTestRunner
 </pre>

 <p>To run only a single method in a test class, append a pound (#) sign and the
 method name (in this case, <code>testConnectPro</code>) to the class name, like so:</p>

 <pre class="devsite-terminal devsite-click-to-copy">
 adb shell am instrument -e class 'com.android.mediaframeworktest.integration.CameraBinderTest#testConnectPro' -w com.android.mediaframeworktest/.MediaFrameworkIntegrationTestRunner
 </pre>

 <h3 id=media_settings_functional_tests>Media settings functional tests</h3>

 <p>Here is an example run of a functional test. This test verifies the basic
 functionality of different combinations of camera settings. (ie, Flash,
 exposure, WB, scene, picture size and geoTag)</p>

 <p>Run the test command:</p>
 <pre class="devsite-terminal devsite-click-to-copy">
 adb shell am instrument -w -r  -e delay_msec 15 -e log true -e class com.android.mediaframeworktest.functional.camera.CameraPairwiseTest com.android.mediaframeworktest/com.android.mediaframeworktest.CameraStressTestRunner
 </pre>

 <h3 id=media_integration_tests>Media integration tests</h3>

 <p>Here is an example run of an integration test, in this case
 mediaframeworktest/integration/CameraBinderTest.java and
 mediaframeworktest/CameraStressTestRunner.java:</p>

 <pre class="devsite-terminal devsite-click-to-copy">
 adb  shell am instrument -e class \ 'com.android.mediaframeworktest.<strong>integration</strong>.<strong>CameraBinderTest'</strong> -w \ 'com.android.mediaframeworktest/.<strong>CameraStressTestRunner'</strong>
 </pre>

 <p>If successful, this results in output resembling:</p>

 <pre class="devsite-click-to-copy">
 -----

 com.android.mediaframeworktest.integration.CameraBinderTest:...........
 Test results for CameraStressTestRunner=...........
 Time: 3.328

 OK (11 tests)

 -----
 </pre>

 <h3 id=media_performance_tests>Media performance tests</h3>

 <p>This preview memory test will open and release the camera preview for 200
 times. In each 20 iterations, the snapshot of ps mediaserver will be recorded
 and it will compare the memory usage different after 200 iterations. Test will
 fail If the difference is greater than 150kM.</p>

 <p>Run the test command:</p>
 <pre class="devsite-terminal devsite-click-to-copy">
 adb shell am instrument -w -r  -e class com.android.mediaframeworktest.performance.MediaPlayerPerformance#testCameraPreviewMemoryUsage com.android.mediaframeworktest/.MediaFrameworkPerfTestRunner
 </pre>

 <p>More detailed output can be found in:
 <code>/sdcard/mediaMemOutput.txt</code></p>

 <h3 id=media_unit_tests>Media unit tests</h3>

 <p>The commands to run unit tests are all similar. For example, for
 CameraMetadataTest.java, the command would be:</p>

 <pre class="devsite-terminal devsite-click-to-copy">
 adb shell am instrument -e class 'com.android.mediaframeworktest.unit.CameraMetadataTest' -w 'com.android.mediaframeworktest/.CameraStressTestRunner'
 </pre>

 <h3 id=media_stress_tests>Media stress tests</h3>

 <p>This test is to stress out the camera image capture and video recording.</p>

 <p>Run the test command:</p>

 <pre class="devsite-terminal devsite-click-to-copy">
 adb shell am instrument -w com.google.android.camera.tests/com.android.camera.stress.CameraStressTestRunner
 </pre>

 <p>All tests should pass.</p>

 <h2 id=manual_testingcam_tests>Manual TestingCam tests</h2>

 <p>The TestingCam app should be run manually with the following checks performed.
 The source for TestingCam is here: <code>pdk/apps/TestingCamera/</code></p>

 <h3 id=infinity_focus_with_camera_tilt>Infinity focus with camera tilt</h3>

 <p>Start TestingCam, turn on preview, and ensure that autofocus mode is set to
 infinity. Using the <strong>Take picture</strong> button, capture shots of
 distant subjects (at least 10m away) with the camera pointed horizontally,
 upwards (close to vertical), and downwards (close to vertical); an example of
 the upwards shot could be high leaves/branches of a tree from beneath and an
 example of the downwards shot could be the street as seen from the roof of a
 building. In all cases, the distant subject should be sharp and in focus. Save
 and view the shots in the gallery view so that you can zoom in and inspect the
 sharpness more easily.</p>

 <p>Note that for a camera with a VCM actuator to pass this test, it will require
 either a closed-loop AF control system, or it will need some sort of SW
 correction based on using accelerometer data to determine camera orientation.
 Reliable factory calibration of the lens infinity position will also be needed.</p>

 <h2 id=manual_testingcam2_tests>Manual TestingCam2 tests</h2>

 <p>The TestingCam2 app should be run manually, with the following checks
 performed. The source for TestingCam2 is here: <code>pdk/apps/TestingCamera2/</code></p>

 <h3 id=9_1_jpeg_capture>JPEG capture</h3>

 <p>Start TestingCam2, and press the <strong>JPEG</strong> button. The image
 that appears to the right of the viewfinder image should appear the same as the
 viewfinder, including having the same orientation.</p>

   </body>
 </html>
	<html devsite>
	<head>
	<title>Camera HAL Testing Checklist</title>
	<meta name="project_path" value="/_project.yaml" />
	<meta name="book_path" value="/_book.yaml" />
	</head>
	<body>
	<!--
	Copyright 2017 The Android Open Source Project

	Licensed under the Apache License, Version 2.0 (the "License");
	you may not use this file except in compliance with the License.
	You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

	Unless required by applicable law or agreed to in writing, software
	distributed under the License is distributed on an "AS IS" BASIS,
	WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
	See the License for the specific language governing permissions and
	limitations under the License.
	-->

	{% include "_versions.html" %}

	<p>This document lists all tests available for evaluating the Android camera
	hardware abstraction layer (HAL). It is intended for original equipment
	manufacturers (OEMs) and application
	processor (AP) vendors so they may ensure proper implementation of the camera
	HAL with minimum defects. Although this is a voluntary addition to the Android
	Compatibility Test Suite (CTS), it greatly increases camera test coverage and
	will certainly identify potential bugs.</p>

	<p>By passing these tests, OEMs validate
	whether they have properly integrated the Android
	camera hardware
	abstraction layer (HAL) 3 interfaces. When conforming with all items in the
	checklist, a device implementation may be considered <em>full</em> with respect
	to the Android Camera HAL interfaces. This will in turn enable a device to
	properly support the
	<a href="https://developer.android.com/reference/android/hardware/camera2/package-summary" class="external">
	<code>android.hardware.camera2</code></a> package that
	camera apps build upon.</p>

	<h2 id=camera_hal_3_2_specification>Camera HAL3 specification</h2>

	<p>The <a href="/devices/camera/camera3">Android Camera HAL3</a> specification
	is the authoritative source of
	information on what devices must satisfy; this page provides a summary
	of all tests that can be used as a checklist. Camera HAL implementers (e.g. AP
	vendors) should go through the Camera HAL3 specification line-by-line and ensure
	their devices conform to it.</p>

	<p>The current HAL specification is defined in these files within the Android
	5.0 and later generic Android Platform Development Kit (PDK):</p>

	<ul>
	<li><em>Camera HAL 3.x interface and spec</em>: <code><a
	href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/camera3.h"
	class="external">hardware/libhardware/include/hardware/camera3.h</a></code>,
	<code><a
	href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/camera_common.h"
	class="external">hardware/libhardware/include/hardware/camera_common.h</a></code>
	<li><em>Camera HAL 3.x metadata spec</em>: <code><a
	href="https://android.googlesource.com/platform/system/media/+/master/camera/docs/docs.html"
	class="external">system/media/camera/docs/docs.html</a></code>
	<li><em>HAL pixel format interface and spec</em>: <code><a
	href="https://android.googlesource.com/platform/system/core/+/master/libsystem/include/system/graphics.h"
	class="external">system/core/libsystem/include/system/graphics.h</a></code>
	</ul>

	<h2 id=camera_test_types>Camera test types</h2>

	<p>Here are the primary types of tests available for the latest Android camera
	along with references to associated instructions below:</p>

	<ul>
	<li><em><a href="#vendor_test_suite">Vendor Test Suite (VTS)</a>:</em> Tests that directly test the
	camera HAL interface
	<li><em><a href="#cts_tests">Compatibility Test Suite (CTS)</a></em>:
	Standard, automated Android
	tests to ensure device compatibility - see the
	<a href="/compatibility/cts/index.html">CTS introduction</a> and the <a
	href="/devices/tech/test_infra/tradefed/index.html">Trade Federation
	Overview</a>
	<li><em><a href="#its_tests">Image Test Suite (ITS)</a>:</em> Manually run
	tests to ensure image correctness - see the top-level and test-specific
	<code>README</code> files and <code>tutorial.py</code> for setup instructions
	<li><em><a
	href="#manual_testingcam_tests">Manual TestingCam tests</a>:</em>
	Run from the source in <code>pdk/apps/TestingCamera/</code>
	<li><em><a href="#manual_testingcam2_tests">Manual TestingCam2.1
	tests</a>:</em> Run from the source in <code>pdk/apps/TestingCamera2/</code>
	</ul>

	<p>All of these test types are described in detail below. These tests are
	presented in the chronological order in which OEMs are expected to execute
	them.</p>

	<p>For instance, if a device fails the native tests, it will assuredly fail the
	subsequent Compatibility Test Suite (CTS) tests. And if a device fails CTS,
	there is little use in proceeding to the Image Test Suite (ITS). We recommend
	addressing failures in each test type before proceeding to the next set of
	tests.</p>

	<h2 id=vendor_test_suite>Vendor Test Suite (VTS) tests</h2>

	<p>The Android Vendor Test Suite (VTS) is a testing suite that works on the
	HIDL interface level. For more information on using VTS, see
	<a href="/compatibility/vts/">Vendor Test Suite</a>.</p>

	<h2 id=cts_tests>Compatibility Test Suite (CTS) tests</h2>

	<p>Camera Android Compatibility Test Suite (CTS) tests focus upon device
	compatibility. For more information on the recommended test environment for camera CTS, see <a href="/compatibility/cts/setup#camera">Set up CTS</a>.</p>

	<p>The starting path for Camera CTS tests is: <code>platform/cts</code>.</p>

	<p>When running Camera CTS for devices that support external cameras (such as
	USB webcams), you must have a device plugged in when running CTS or the tests
	will automatically fail. Examples of external cameras include: <a
	href="https://www.logitech.com/en-us/product/hd-pro-webcam-c920"
	class="external">Logitech HD Pro Webcam C920</a> and the <a
	href="https://www.microsoft.com/accessories/en-us/products/webcams/lifecam-hd-3000/t3h-00011"
	class="external">Microsoft LifeCam HD-3000</a>.</p>

	<p>See the <a href="/compatibility/cts/index.html">CTS
	introduction</a> and its subpages for general instructions on running CTS.</p>

	<h3 id=cts_tests_for_the_android_hardware_camera_api>CTS tests for
	the <code>android.hardware.Camera</code> API</h3>

	<p>Find these camera tests under <code>cts/tests/tests/</code>:</p>

	<ul>
	<li><code>hardware/src/android/hardware/cts/CameraTest.java</code>
	<li><code>hardware/src/android/hardware/cts/CameraGLTest.java</code>
	<li><code>hardware/src/android/hardware/cts/Camera_SizeTest.java</code>
	<li><code>permission/src/android/permission/cts/CameraPermissionTest.java</code>
	</ul>

	<h3 id=cts_tests_for_the_android_hardware_camera2_api>CTS tests for
	the <code>android.hardware.camera2</code> API</h3>

	<p>Find these camera tests under <code>cts/tests/tests/</code>:</p>

	<ul>
	<li><code>hardware/src/android/hardware/camera2/cts/*</code>
	<li><code>permission/src/android/permission/cts/Camera2PermissionTest.java</code>
	</ul>

	<h3 id=cts_verifier_camera_tests>CTS Verifier camera tests</h3>

	<p>Find these camera tests under:
	<code>cts/apps/CtsVerifier/src/com/android/cts/verifier/camera/*</code></p>

	<h2 id=its_tests>Image Test Suite (ITS) tests</h2>

	<aside class="Note"><strong>Note:</strong>
	<a href="/devices/camera/camera3">Camera HAL3</a> is strongly recommended for
	devices running Android {{ androidPVersionNumber }} or higher as Camera HAL1
	has been deprecated. This
	also allows for a smooth phase out of the deprecated
	<a href="https://developer.android.com/reference/android/hardware/Camera" class="external">
	Camera API</a>.</aside>

	<p>The Camera Image Test Suite (ITS) tests focus on image correctness.
	To perform the tests, run the Python scripts on a workstation with the
	Android device connected over USB.</p>

	<p>The Camera ITS infrastructure and tests are located in the
	<a href="https://android.googlesource.com/platform/cts/+/master/apps/CameraITS"
	class="external">
	<code>cts/apps/CameraITS</code></a> directory. Each test resides in a
	<code>tests/scene<var>#</var></code> subdirectory.</p>

	To set up the test environment, run:

	<pre class="devsite-click-to-copy">
	<code class="devsite-terminal">extract root/out/host/linux-x86/cts-verfier/android-cts-verifier.zip</code>
	<code class="devsite-terminal">cd android-cts-verifier</code>
	<code class="devsite-terminal">adb install -r CtsVerifier.apk</code>
	<code class="devsite-terminal">cd CameraITS</code>
	<code class="devsite-terminal">source build/envsetup.sh</code>
	</pre>

	<aside class="note"><strong>Note:</strong> Because ITS is a CTS Verifier subtest,
	start CTS Verifier and the ITS subtest before running the Python scripts so they
	have processes with which to communicate.</aside>

	<p>For more information on how to set up and run the tests, see the
	<code>CameraITS</code> PDF file in <code>cts/apps/CameraITS</code>. See
	<a href="https://android.googlesource.com/platform/cts/+/master/apps/CameraITS/tests/tutorial.py"
	class="external">
	<code>tutorial.py</code></a> in the <code>tests</code> subdirectory for a
	guide on how to use the scripts.</p>

	<p>The ITS static tests (scenes 0-5) can run with any operating
	system with the required Python 2.7 environment. However, the
	<code>sensor_fusion</code>
	test with the <a href="/compatibility/cts/sensor-fusion-quick-start">Sensor
	Fusion Box</a> must be run with the
	Linux operating system.
	</p>

	<p>The recommended setup for scenes 0-4
	is described in <a
	href="/compatibility/cts/camera-its-box">Camera ITS-in-a-box</a>. The
	recommended setup for the sensor_fusion scene is described in <a
	href="/compatibility/cts/sensor-fusion-quick-start">Sensor Fusion Box Quick
	Start Guide</a>.</p>

	<p>To run ITS manually, prepare a simple physical environment with a
	specific, reusable target
	such as a white wall, a grey card, and a desk lamp. Mount the Android device on
	a tripod and run the scripts to test the camera functions. Most tests
	are pass or fail but some offer metrics.</p>

	<p>These scripts test scenarios that are not tested in CTS and are an important
	component of the HAL 3.2 test plan.</p>

	<p>ITS tests either pass or fail. All mandated
	tests in each scene folder must pass. Tests that are not mandated can
	fail and still count as a pass in
	<code>CtsVerifier</code>.</p>

	<h3 id="scene0_to_scene4_tests">Scene 0 to scene 4 tests</h3>

	<p>These scenes represent a large part of ITS testing and are included as PDF
	files in the <code>scene</code> folder. To automate these tests, use the <a
	href="/compatibility/cts/camera-its-box">Camera ITS-in-a-box</a> system.</p>

	<ul>
	<li>Scene 0: Requires no setup.</li>
	<li>Scene 1: Requires a grey card.</li>
	<li>Scene 2: Requires a face scene.</li>
	<li>Scene 3: ISO12233 chart.</li>
	<li>Scene 4: Custom scene with a circle inside a square.</li>
	</ul>

	<h3 id="scene5_tests">Scene 5 tests</h3>
	<p>Scene 5 tests require a diffuser to be placed on top of the camera.</p>

	<h3 id="sensor_fusion_tests">Sensor fusion tests</h3>

	<p>Sensor fusion tests require specific camera motion to test the timestamp
	difference between the camera and the gyroscope for AR and VR applications. This
	test is skipped if no gyroscope is included or if the <code>REALTIME</code>
	parameter is not enabled. The <code>sensor_fusion</code> test can be automated
	with the <a
	href="/compatibility/cts/sensor-fusion-quick-start">Sensor Fusion Box</a>.</p>

	<h2 id=media_framework_tests>Media Framework tests</h2>

	<p>Pass all of the camera-related media tests in MediaFrameworkTest. Please
	note, these tests require the mediaframeworktest.apk be installed on the Android
	device. You will need to <code>make mediaframeworktest</code> and then use adb
	to install the resulting .apk. Example commands are included below.</p>

	<p>The starting path for Camera-related media framework tests is:
	<code>platform/frameworks/base</code></p>

	<p>Find the source code for the tests here:
	<code>frameworks/base/media/tests/MediaFrameworkTest</code></p>

	<p>To set up these tests:</p>

	<pre class="devsite-click-to-copy">
	<code class="devsite-terminal">make mediaframeworktest</code>
	<code class="devsite-terminal">adb install out/target/product/<var>name</var>/data/app/mediaframeworktest.apk</code>
	</pre>

	<p>Where the <var>name</var> variable represents the
	directory containing the vendor's product.</p>

	<p>Find all of the tests in the following directory or its subdirectories:</p>

	<pre class="devsite-click-to-copy">
	frameworks/base/media/tests/MediaFrameworkTest/src/com/android/mediaframeworktest
	</pre>

	<p>Each subdirectory represents a class of tests:</p>

	<ul>
	<li><code>functional/</code>
	<li><code>integration/</code>
	<li><code>performance/</code>
	<li><code>power/</code>
	<li><code>stress/</code>
	<li><code>unit/</code>
	</ul>

	<h3 id=running_media_framework_tests>Running Media Framework tests</h3>

	<p>To see all of the available tests::</p>

	<pre class="devsite-terminal devsite-click-to-copy">
	adb shell pm list instrumentation
	</pre>

	<p>This will yield results resembling:</p>

	<pre class="devsite-click-to-copy">
	instrumentation:com.android.mediaframeworktest/.MediaFrameworkIntegrationTestRunner
	(target=com.android.mediaframeworktest)
	instrumentation:com.android.mediaframeworktest/.MediaRecorderStressTestRunner
	(target=com.android.mediaframeworktest)
	instrumentation:com.android.mediaframeworktest/.MediaFrameworkPerfTestRunner
	(target=com.android.mediaframeworktest)
	instrumentation:com.android.mediaframeworktest/.MediaFrameworkPowerTestRunner
	(target=com.android.mediaframeworktest)
	</pre>

	<p>Identify and extract the component (between <code>instrumentation:</code>
	and <code>(target=com.android.mediaframeworktest) </code>from each test line.
	The component is composed of the target package name
	(<code>com.android.mediaframeworktest</code>) and the test runner name
	(<code>MediaFramework<type>TestRunner</code>).</p>

	<p>For instance:</p>

	<pre class="devsite-click-to-copy">
	com.android.mediaframeworktest/.MediaFrameworkIntegrationTestRunner
	com.android.mediaframeworktest/.MediaRecorderStressTestRunner
	com.android.mediaframeworktest/.MediaFrameworkPerfTestRunner
	com.android.mediaframeworktest/.MediaFrameworkPowerTestRunner
	</pre>

	<p>You may then pass each component to <code>adb shell am instrument</code> like
	so:</p>

	<pre class="devsite-terminal devsite-click-to-copy">
	adb shell am instrument -w <var>component.name</var>
	</pre>

	<p>Where <code><var>component.name</var></code> equals the extracted value
	above. For
	example:</p>

	<pre class="devsite-terminal devsite-click-to-copy">
	adb shell am instrument -w com.android.mediaframeworktest/.MediaFrameworkIntegrationTestRunner
	</pre>

	<p>Please note, while the class path is the Java package + class name, the
	instrumentation package isn't necessarily the same as the Java package. Make
	sure you use the AndroidManifest.xml package when concatenating the component
	name, not the Java package in which the test runner class resides.</p>

	<p>To run a single class of tests, pass the -e class <test-class> argument, like
	so:</p>

	<pre class="devsite-terminal devsite-click-to-copy">
	adb shell am instrument -e class com.android.mediaframeworktest.integration.CameraBinderTest -w com.android.mediaframeworktest/.MediaFrameworkIntegrationTestRunner
	</pre>

	<p>To run only a single method in a test class, append a pound (#) sign and the
	method name (in this case, <code>testConnectPro</code>) to the class name, like so:</p>

	<pre class="devsite-terminal devsite-click-to-copy">
	adb shell am instrument -e class 'com.android.mediaframeworktest.integration.CameraBinderTest#testConnectPro' -w com.android.mediaframeworktest/.MediaFrameworkIntegrationTestRunner
	</pre>

	<h3 id=media_settings_functional_tests>Media settings functional tests</h3>

	<p>Here is an example run of a functional test. This test verifies the basic
	functionality of different combinations of camera settings. (ie, Flash,
	exposure, WB, scene, picture size and geoTag)</p>

	<p>Run the test command:</p>
	<pre class="devsite-terminal devsite-click-to-copy">
	adb shell am instrument -w -r -e delay_msec 15 -e log true -e class com.android.mediaframeworktest.functional.camera.CameraPairwiseTest com.android.mediaframeworktest/com.android.mediaframeworktest.CameraStressTestRunner
	</pre>

	<h3 id=media_integration_tests>Media integration tests</h3>

	<p>Here is an example run of an integration test, in this case
	mediaframeworktest/integration/CameraBinderTest.java and
	mediaframeworktest/CameraStressTestRunner.java:</p>

	<pre class="devsite-terminal devsite-click-to-copy">
	adb shell am instrument -e class \ 'com.android.mediaframeworktest.<strong>integration</strong>.<strong>CameraBinderTest'</strong> -w \ 'com.android.mediaframeworktest/.<strong>CameraStressTestRunner'</strong>
	</pre>

	<p>If successful, this results in output resembling:</p>

	<pre class="devsite-click-to-copy">
	-----

	com.android.mediaframeworktest.integration.CameraBinderTest:...........
	Test results for CameraStressTestRunner=...........
	Time: 3.328

	OK (11 tests)

	-----
	</pre>

	<h3 id=media_performance_tests>Media performance tests</h3>

	<p>This preview memory test will open and release the camera preview for 200
	times. In each 20 iterations, the snapshot of ps mediaserver will be recorded
	and it will compare the memory usage different after 200 iterations. Test will
	fail If the difference is greater than 150kM.</p>

	<p>Run the test command:</p>
	<pre class="devsite-terminal devsite-click-to-copy">
	adb shell am instrument -w -r -e class com.android.mediaframeworktest.performance.MediaPlayerPerformance#testCameraPreviewMemoryUsage com.android.mediaframeworktest/.MediaFrameworkPerfTestRunner
	</pre>

	<p>More detailed output can be found in:
	<code>/sdcard/mediaMemOutput.txt</code></p>

	<h3 id=media_unit_tests>Media unit tests</h3>

	<p>The commands to run unit tests are all similar. For example, for
	CameraMetadataTest.java, the command would be:</p>

	<pre class="devsite-terminal devsite-click-to-copy">
	adb shell am instrument -e class 'com.android.mediaframeworktest.unit.CameraMetadataTest' -w 'com.android.mediaframeworktest/.CameraStressTestRunner'
	</pre>

	<h3 id=media_stress_tests>Media stress tests</h3>

	<p>This test is to stress out the camera image capture and video recording.</p>

	<p>Run the test command:</p>

	<pre class="devsite-terminal devsite-click-to-copy">
	adb shell am instrument -w com.google.android.camera.tests/com.android.camera.stress.CameraStressTestRunner
	</pre>

	<p>All tests should pass.</p>

	<h2 id=manual_testingcam_tests>Manual TestingCam tests</h2>

	<p>The TestingCam app should be run manually with the following checks performed.
	The source for TestingCam is here: <code>pdk/apps/TestingCamera/</code></p>

	<h3 id=infinity_focus_with_camera_tilt>Infinity focus with camera tilt</h3>

	<p>Start TestingCam, turn on preview, and ensure that autofocus mode is set to
	infinity. Using the <strong>Take picture</strong> button, capture shots of
	distant subjects (at least 10m away) with the camera pointed horizontally,
	upwards (close to vertical), and downwards (close to vertical); an example of
	the upwards shot could be high leaves/branches of a tree from beneath and an
	example of the downwards shot could be the street as seen from the roof of a
	building. In all cases, the distant subject should be sharp and in focus. Save
	and view the shots in the gallery view so that you can zoom in and inspect the
	sharpness more easily.</p>

	<p>Note that for a camera with a VCM actuator to pass this test, it will require
	either a closed-loop AF control system, or it will need some sort of SW
	correction based on using accelerometer data to determine camera orientation.
	Reliable factory calibration of the lens infinity position will also be needed.</p>

	<h2 id=manual_testingcam2_tests>Manual TestingCam2 tests</h2>

	<p>The TestingCam2 app should be run manually, with the following checks
	performed. The source for TestingCam2 is here: <code>pdk/apps/TestingCamera2/</code></p>

	<h3 id=9_1_jpeg_capture>JPEG capture</h3>

	<p>Start TestingCam2, and press the <strong>JPEG</strong> button. The image
	that appears to the right of the viewfinder image should appear the same as the
	viewfinder, including having the same orientation.</p>

	</body>
	</html>