tree b4227f625f29e2426693f940545b2883b51fbe5d
parent 2a75dcb2fd1490ace0acf82e948aae2829b87e8d
author Paul Duffin <paulduffin@google.com> 1495710189 +0100
committer Paul Duffin <paulduffin@google.com> 1495818581 +0100

Fix JUnitParamsRunner so it works with CTS sharding

CTS uses the Android Test Support Library's
TestRequestBuilder.ShardingFilter to spread changes across a
number of devices in order to parallelize testing.

CTS runs the tests in two modes, in the first it collects the
set of tests that will be run - the list of tests returned by
Runner.getDescription() (after filtering) and in the second it
actually runs the tests as returned by
ParentRunner.getFilteredChildren().

JUnitParams does not work in that situation because it applies
the filter in a different way for parameterized methods
depending on whether it is collecting or running which leads
to inconsistent methods in each phase which causes CTS problems.

When collecting it creates a flat list of FrameworkMethod
instances, one for each method, whether parameterized or
not and applies the filter to that. Once it has filtered it
iterates over to create the Description and during that
process it creates N Description objects for each parameterized
method and 1 Description object for each non-parameterized
method. That means that for each parameterized method either
every instance is collected or none of them are.

When running it creates a flat list of FrameworkMethod
instances, one for each non-parameterized method and N for each
parameterized method (where N is the number of parameter sets
supplied for the method). They are then filtered individually.
That means that for each parameterized method some of its
instances are run but not necessarily all.

This fixes it by making the running and describing parts
completely consistent in how they apply the filters.

This change will be pushed upstream if possible.

Tested by running the two commands given in the bug and ensuring
that they produce the correct set of tests. Added target to
build the test on host and ran selected tests from there. Ran
all tests on the device as per instructions in Android.mk file.

Bug: 38419944
Test: See above
Change-Id: I25b4d4130ffdc71c77992abf592662ba1e1432db