commit | fac6039ff464d0bbf02d42d187073f5e1cec59be | [log] [tgz] |
---|---|---|
author | Ang Li <xpconanfan@users.noreply.github.com> | Wed Dec 07 14:30:35 2016 -0800 |
committer | GitHub <noreply@github.com> | Wed Dec 07 14:30:35 2016 -0800 |
tree | c18a3eda552eec9fc5435254cea35e121f5c8766 | |
parent | f0151687f21f848928f491cb927fddcf488ae2fa [diff] |
Add an arg to check the number of controller objects (#10) Add an arg in TestRunner.register_controller to check the number of controller objects meets minimum requirement. Also clean up unit tests for register_controller: * Add a test for the new arg * Remove an old test for the now-removed builtin controller code path * Rename some tests to match removing the builtin controller code path * Remove an implementation leftover check.
What is Mobly?
Mobly is a Python-based test framework that specializes in supporting test cases that require multiple devices, complex environments, or custom hardware setups. Examples:
Mobly can support many different types of devices and equipment, and it's easy to plug your own device or custom equipment/service into Mobly.
Mobly comes with a set of libs to control common devices like Android devices.
While developed by Googlers, Mobly is not an official Google product.
What will I learn here?
Writing and executing simple test cases that use Android devices. We are focusing on Android devices here since they are the most accessible devices. Mobly supports various devices and you can also use your own custom hardware/equipment.
Before you start, you need:
adb devices
.* You can get SL4A from the Android repo, under project <aosp>/external/sl4a
It can be built like a regular system app with mm
commands. It needs to be signed with the build you use on your Android devices.
Python compatibility
Mobly is compatible with both python 3.4+ and python 2.7.
System dependencies
Let‘s start with the simple example of posting “Hello World” on the Android device’s screen. Create the following files:
sample_config.json
{ "testbed": [ { "_description": "A testbed where adb will find Android devices.", "name": "SampleTestBed", "AndroidDevice": "*" } ], "logpath": "/tmp/logs" }
hello_world_test.py
from mobly import base_test from mobly import test_runner from mobly.controllers import android_device class HelloWorldTest(base_test.BaseTestClass): def setup_class(self): self.ads = self.register_controller(android_device) self.dut = self.ads[0] def test_hello(self): self.dut.sl4a.makeToast('Hello World!') if __name__ == "__main__": test_runner.main()
To execute:
$ python hello_world_test.py -c sample_config.json
Expect
A “Hello World!” toast notification appears on your device's screen.
Within SampleTestBed, we used "AndroidDevice" : "*"
to tell the test runner to automatically find all connected Android devices. You can also specify particular devices by serial number and attach extra attributes to the object:
"AndroidDevice": [ {"serial": "xyz", "phone_number": "123456"}, {"serial": "abc", "label": "golden_device"}, ]
We have multiple tests written in a test script, and we only want to execute a subset of them.
hello_world_test.py
from mobly import base_test from mobly import test_runner from mobly.controllers import android_device class HelloWorldTest(base_test.BaseTestClass): def setup_class(self): self.ads = self.register_controller(android_device) self.dut = self.ads[0] def test_hello(self): self.dut.sl4a.makeToast('Hello World!') def test_bye(self): self.dut.sl4a.makeToast('Goodbye!') if __name__ == "__main__": test_runner.main()
To execute:
$ python hello_world_test.py -c sample_config.json --test_case test_bye
Expect
A “Goodbye!” toast notification appears on your device's screen.
You can dictate what test cases to execute within a test script and their execution order, shown below:
$ python hello_world_test.py -c sample_config.json --test_case test_bye test_hello test_bye
Expect
Toast notifications appear on your device's screen in the following order: “Goodbye!”, “Hello World!”, “Goodbye!”.
You could specify user parameters to be passed into your test class in the config file.
In the following config, we added a user parameter favorite_food
.
sample_config.json
{ "testbed": [ { "name" : "SampleTestBed", "AndroidDevice" : "*" } ], "logpath" : "/tmp/logs", "favorite_food" : "Green eggs and ham." }
In the test script, you could access the user parameter:
def test_favorite_food(self): food = self.user_params.get('favorite_food') if food: self.dut.sl4a.makeToast("I'd like to eat %s." % food) else: self.dut.sl4a.makeToast("I'm not hungry.")
Multiple test beds can be configured in one configuration file.
sample_config.json
{ "testbed":[ { "name" : "XyzTestBed", "AndroidDevice" : [{"serial": "xyz", "phone_number": "123456"}] }, { "name" : "AbcTestBed", "AndroidDevice" : [{"serial": "abc", "label": "golden_device"}] } ], "logpath" : "/tmp/logs", "favorite_food": "green eggs and ham" }
You can choose which one to execute on with the command line argument --test_bed
:
$ python hello_world_test.py -c sample_config.json --test_bed AbcTestBed
Expect
A “Hello World!” and a “Goodbye!” toast notification appear on your device's screen.
In this example, we use one Android device to discover another Android device via bluetooth. This test demonstrates several essential elements in test writing, like logging and asserts.
sample_config.json
{ "testbed":[ { "name" : "TwoDeviceTestBed", "AndroidDevice" : [{"serial": "xyz", "label": "dut"}, {"serial": "abc", "label": "discoverer"}] } ], "logpath" : "/tmp/logs", "bluetooth_name": "MagicBluetooth", "bluetooth_timeout": 5 }
sample_test.py
from mobly import base_test from mobly import test_runner from mobly.controllerse import android_device class HelloWorldTest(base_test.BaseTestClass): def setup_class(self): self.ads = self.register_controller(android_device) self.dut = android_device.get_device(self.ads, label="dut") self.discoverer = android_device.get_device(self.ads, label="discoverer") self.dut.ed.clear_all_events() self.discoverer.ed.clear_all_events() def setup_test(self): # Make sure bluetooth is on self.dut.sl4a.bluetoothToggleState(True) self.discoverer.sl4a.bluetoothToggleState(True) self.dut.ed.pop_event(event_name='BluetoothStateChangedOn', timeout=10) self.discoverer.ed.pop_event(event_name='BluetoothStateChangedOn', timeout=10) if (not self.dut.sl4a.bluetoothCheckState() or not self.discoverer.sl4a.bluetoothCheckState()): asserts.abort_class('Could not turn on Bluetooth on both devices.') # Set the name of device #1 and verify the name properly registered. self.dut.sl4a.bluetoothSetLocalName(self.bluetooth_name) asserts.assert_equal(self.dut.sl4a.bluetoothGetLocalName(), self.bluetooth_name, 'Failed to set bluetooth name to %s on %s' % (self.bluetooth_name, self.dut.serial)) def test_bluetooth_discovery(self): # Make dut discoverable. self.dut.sl4a.bluetoothMakeDiscoverable() scan_mode = self.dut.sl4a.bluetoothGetScanMode() asserts.assert_equal( scan_mode, 3, # 3 signifies CONNECTABLE and DISCOVERABLE 'Android device %s failed to make blueooth discoverable.' % self.dut.serial) # Start the discovery process on #discoverer. self.discoverer.ed.clear_all_events() self.discoverer.sl4a.bluetoothStartDiscovery() self.discoverer.ed.pop_event( event_name='BluetoothDiscoveryFinished', timeout=self.bluetooth_timeout) # The following log entry demonstrates AndroidDevice log object, which # prefixes log entries with "[AndroidDevice|<serial>] " self.discoverer.log.info('Discovering other bluetooth devices.') # Get a list of discovered devices discovered_devices = self.discoverer.sl4a.bluetoothGetDiscoveredDevices() self.discoverer.log.info('Found devices: %s', discovered_devices) matching_devices = [d for d in discovered_devices if d.get('name') == self.bluetooth_name] if not matching_devices: asserts.fail('Android device %s did not discover %s.' % (self.discoverer.serial, self.dut.serial)) self.discoverer.log.info('Discovered at least 1 device named ' '%s: %s', self.bluetooth_name, matching_devices) if __name__ == "__main__": test_runner.main()
One will notice that this is not the most robust test (another nearby device could be using the same name), but in the interest of simplicity, we've limited the number of RPCs sent to each Android device to just two:
self.dut
, we asked it to make itself discoverable and checked that it did it.self.discoverer
, we asked it to start scanning for nearby bluetooth devices, and then we pulled the list of devices seen.There's potentially a lot more we could do to write a thorough test (e.g., check the hardware address, see whether we can pair devices, transfer files, etc.).
You‘ll notice above that we’ve used self.{device_alias}.ed.pop_event()
. The ed
attribute of an Android device object is an EventDispatcher, which provides APIs to interact with async events.
For example, pop_event
is a function which will block until either a specified event is seen or the call times out, and by using it we avoid the use of busy loops that constantly check the device state. For more, see the APIs in mobly.controllers.android_device_lib.event_dispatcher
.