January 9, 2013

223 Views

An Overview of the Android Automation Testing Landscape

The booming Android application market has posed several challenges to software testing, which are compounded by the reduced time available for each test cycle. Some of the factors responsible for these challenges are:

  • The diversity of Android-based handsets in terms of screen size, OEM, operators. etc.

  • Multiple devices in the market with different versions of Android OS

  • The upgraded version (typically there is a new Android release every 6-7 months)

Overcoming these challenges is an opportunity for software testing to reach to the next level of testing, i.e. fully automated testing.  Android has provided frameworks and tools for automated testing, and  these tools can be broadly categorized as:

  • Instrumentation-based test framework

  • GUI-based testing tools

  • Third party cloud-based testing services

Various derivatives of these tools are available in the market as open source or proprietary tools.

Instrumentation-Based Test Framework

Android has provided a built-in support to test Android applications using instrumentation.  This support is an extension to the JUnit framework.  It provides control methods or hooks that can:

  • Control an Android component independently of its normal lifecycle

  • Control over how Android loads applications

  • Load test package and the application under test into the same process.

It requires some information about the source code. Using instrumentation, various vendors have developed their own frameworks that don’t require source code, e.g. Robotium, Calculon, Positron, etc.

Instrumentation-based automation testing is generally used for unit test, Android component testing, activity, application lifecycle testing or business logic testing. Android has also provided extension to JUnit test-case to test all four parts of Android application activity, service, context provider and application.

GUI Based Testing Tools

Apart from instrumentation framework, Android also provides tools for GUI testing of Applications.

Monkey

This is used to send a series of random events to the emulator and watch the system for crashes or system not responding events. It is generally used as a stress testing tool. Using various command line parameters, the tool can be used for stress testing, debugging and analyzing crashes.

MonkeyRunner

This provides the API for writing programs that can control an Android device or emulator from outside of Android code. The monkeyrunner tool uses Jython, an implementation of Python that uses the Java programming language. Jython allows the monkeyrunner API to interact easily with the Android framework. Using jython and monkeyrunner, a user can install an Android application, send keystrokes to it, take screenshots of the UI and store them on the workstation. It is used to test the application at the framework /functional level.

Both Monkey and MonkeyRunner can be used for functional, system, regression and stress testing. The only drawback is that we need to have an image processing tool to verify the test result and a slight modification in the GUI, as even the resolution or screen size will lead to a change in the test images.

Third Party Cloud Based Services

There are also third-party tools that provide Android application testing as a service which typically include a cloud of mobiles, and a web interface to run tests and view test results. Some tools might require a client application to be installed on the system under test, whereas others may provide a facility where developers can record their test or upload their automation test and test their application on a number of devices. This can help OEMs reduce the testing infrastructure and effort while also providing testing across different geographies and network operators.

As we can see, a variety of testing tools is available for Android automation testing. OEMs/application  developers should create a test strategy depending on their testing requirements, current test infrastructure and effort budgets.

Abbreviations

API

Application Pogramming Interface

GUI

Graphics User Interface

OEM

Original Equipment Manufacturer

Definitions

JUnit is a unit testing framework for the Java programming language.

Referneces

http://developer.android.com/tools/testing/testing_android.html

http://code.google.com/p/robotium/

http://brainflush.wordpress.com/2010/01/10/introducing-calculon-a-java-dsl-for-android-activity-testing/

http://code.google.com/p/autoandroid/wiki/Positron             

http://en.wikipedia.org/wiki/JUnit

http://silvercg.com/wp-content/uploads/2011/01/android9_background_21.jpeg