Generate Android testing reports in XML/HTML with Android CTS

When you run automated Android tests with Eclipse or from the command line, you get text output, which isn't good for reporting purposes. If you run a large set
of test cases with automated build, the text report isn't very helpful. Fortunately, Android CTS generates
test reports in XML with accompanying XSL to make it look nice in a browser.

To run your own tests with Android CTS:

Download Android CTS

Make a new directory

MyRepository

under

android-cts

, alongside the existing

repository

directory.

Copy

host_config.xml

from

repository

to

MyRepository

Create directory

plans

under

MyRepository

, add a test plan (

MyTests.xml

):

<?xml version="1.0" encoding="UTF-8"?>
<TestPlan version="1.0">
<Entry uri="android.AndroidTest"/>
</TestPlan>

Create directory

testcases

under

MyRepository

.

Copy

TestDeviceSetup.apk

from

repository/testcases

to

MyRepository/testcases

Under

MyRepository/testcases

, create a test package definition (

AndroidTest.xml

, but can be any name you pick):

<?xml version="1.0" encoding="UTF-8"?>
<TestPackage AndroidFramework="Android 1.0"
appNameSpace="com.android.test"
appPackageName="android.AndroidTest"
name="AndroidTest"
runner="android.test.InstrumentationTestRunner" version="1.0">
<TestSuite name="com">
<TestSuite name="android.test">
<TestCase name="AndroidTest">
<Test name="testCase1"/>
<Test name="testCase2"/>
<Test name="testCase3"/>
<Test name="thisIsNotATest"/>
</TestCase>

<TestCase name="AAndroidTest">
<Test name="testCase1"/>
<Test name="testCase2"/>
</TestCase>
</TestSuite>
</TestSuite>
</TestPackage>

where:

appNameSpace

is the package name of the test suite apk

appPackageName

is the name specified in the test plan

name

is the file name of the test suite apk

Test

are the test methods to call

The

TestSuite

and

TestCase

paths should add up to be the full Java class name of the test class. In fact, the above definition is equivalent to:

<?xml version="1.0" encoding="UTF-8"?>
<TestPackage AndroidFramework="Android 1.0"
appNameSpace="com.android.test"
appPackageName="android.AndroidTest"
name="AndroidTest"
runner="android.test.InstrumentationTestRunner" version="1.0">
<TestSuite name="com">
<TestSuite name="android.test">
<TestCase name="AndroidTest">
<Test name="testCase1"/>
<Test name="testCase2"/>
<Test name="testCase3"/>
<Test name="thisIsNotATest"/>
</TestCase>
</TestSuite>

<TestCase name="android.test.AAndroidTest">
<Test name="testCase1"/>
<Test name="testCase2"/>
</TestCase>
</TestSuite>
</TestPackage>

I created this file manually, but it should be pretty straightforward to write a tool with Java reflection or scripting with regex search if you have the source code.

Copy the test suite apk into

MyRepository/testcases

. The final directory structure looks like this:

$ find MyRepository -type f
MyRepository/plans/MyTests.xml
MyRepository/testcases/AndroidTest.apk
MyRepository/testcases/AndroidTest.xml
MyRepository/testcases/TestDeviceSetup.apk
MyRepository/host_config.xml

Run the tests with

android-cts

:

$ cd android-cts/tools/
$ ./startcts ../MyRepository/host_config.xml
Android CTS version 2.3_r7
cts_host > ls --plan
List of plans (1 in total):
MyTests
cts_host > ls --plan MyTests
Packages of plan MyTests (1 in total):
=================================
android.AndroidTest
cts_host > start --plan MyTests