Final product of Android application development are not Activities, Services, Fragments and Views but simultaneous work of all these pieces to produce system with certain functionalities. Customer and user are not interested in internal architecture of the mobile app but they want to ensure that app returns the correct UI output in response to user’s actions on device. Therefore, functional UI testing does not require testers to know details of implementation.
Manual testing has a lot of disadvantages: it can be time-consuming, tedious and error prone. Automated testing is more efficient and reliable. All you need to do is to write test cases to cover specific usage scenarios, and then run the test cases automatically and repeatedly by testing framework.
Inspiration
The most notable limitation in Android Instrumentation frameworks, including
Robotium, is that it lets click throughout only on the application that is under testing. For example, if application opens the camera and tries to take a photo, the test ends with a fail.
This is because of a permission to perform a click from one application to another. It is related to Android’s security model. For example, the
uiautomator does not have this limitation, it allows taking pictures in one application and enable access to change settings in second application.
Why Appium?
- Provides cross-platform solution for native and hybrid mobile automation i.e. Android and iOS.
- Allows you to communicate with other Android apps not only app under the test. For example, you can start another app from app under the test (for example, Camera app).
- You don’t have to re-compile your app or modify it in any way, due to use of standard automation APIs on all platforms.
- It is “black box”. You can test not only app developed by yourself but any *.apk installed on your phone or emulator. Internal implementation of the app is not limitation for testing (except some rules related to UI definition like defining content-description text).
- You can write tests with your favorite dev tools using any WebDriver compatible language such as Java, Objective-C, JavaScript with node.js, PHP, Ruby, Python, C#… All you need are Selenium WebDriver and language specific libraries.
How it works?
It supports a subset of the
Selenium WebDriver JSON Wire Protocol, and extends it so that user can specify mobile targeted desired capabilities to run tests through
Appium. Android support for Appium uses the UiAutomator framework for newer platforms and
Selendroid for older Android patforms.
Example
My simple example is doing this:
- Runs MainActivity which has a button with label “button1”.
- Clicks on button1 which starts second Activity
- Checks if second screen contains TextView with text “Activity2”
- Clicks on “back” button
- Checks if we are again on MainActivity
public class AppiumExampleTest {
private WebDriver driver = null;
@Before
public void setup() {
File appDir = new File("..//TestedAndroidApp//bin//");
File app = new File(appDir, "TestedAndroidApp.apk");
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.BROWSER_NAME, "");
capabilities.setCapability(CapabilityType.VERSION, "4.2");
capabilities.setCapability(CapabilityType.PLATFORM, "WINDOWS");
capabilities.setCapability(CapabilityConstants.DEVICE, "android");
capabilities.setCapability(CapabilityConstants.APP_PACKAGE, "com.example.android");
capabilities.setCapability(CapabilityConstants.APP_ACTIVITY, "MainActivity");
capabilities.setCapability(CapabilityConstants.APP, app.getAbsolutePath());
try {
driver = new RemoteWebDriver(new URL("http://127.0.0.1:4723/wd/hub"), capabilities);
} catch (MalformedURLException e) {
e.printStackTrace();
}
driver.manage().timeouts().implicitlyWait(80, TimeUnit.SECONDS);
}
@Test
public void appiumExampleTest() throws Exception {
// find button with label or content-description "Button 1"
WebElement button=driver.findElement(By.name("Button 1"));
// click on button and start second Activity
button.click();
// we are on second screen now
// check if second screen contains TextView with text “Activity2”
driver.findElement("Activity2");
// click back button
HashMap<String, Integer> keycode = new HashMap<String, Integer>();
keycode.put("keycode", 4);
((JavascriptExecutor) driver).executeScript("mobile: keyevent", keycode);
// we are again in main activity
driver.findElement(By.name("Button1"));
}
@After
public void tearDown() {
if (driver != null) {
driver.quit();
}
}
}
|
As you can see in code example, we use WebDriver to find elements on UI. It is created in setup() method where we define a set of desired capabilities. When we find certain UI element we can perform some action on it like clicking or type some text in input field.
WebView testing
One feature that is lacking in uiautomator is not existing way to directly access Android objects (Views) and there is a limitation to handle WebView. Because there is not way to access WebView, testers can not inject JavaScript, which is clearly the easiest and the best way to handle those tests. Currently there is nothing testers could do inside WebView with uiautomator.
But Appium developers found solution for this limitation. As Appium has support for both, uiautomator and Selendroid, you can use Selendroid to test WebView. Here is simple example how to do that:
public class LoginTest {
private WebDriver driver = null;
@Before
public void setup() {
File appDir = new File("..//TestedAndroidApp//bin//");
File app = new File(appDir, "TestedAndroidApp.apk");
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.BROWSER_NAME, "");
capabilities.setCapability(CapabilityType.PLATFORM, "WINDOWS");
capabilities.setCapability("device", "selendroid");
capabilities.setCapability(CapabilityConstants.APP_PACKAGE, "com.example.android");
capabilities.setCapability(CapabilityConstants.APP_ACTIVITY, "LoginActivity");
capabilities.setCapability(CapabilityConstants.APP, app.getAbsolutePath());
try {
driver = new RemoteWebDriver(new URL("http://127.0.0.1:4723/wd/hub"), capabilities);
} catch (MalformedURLException e) {
e.printStackTrace();
}
driver.manage().timeouts().implicitlyWait(80, TimeUnit.SECONDS);
}
@Test
public void loginTest() throws Exception {
WebDriverWait wait = new WebDriverWait(driver, 10);
// this is important part.
driver.switchTo().window("WEBVIEW");
// find user-name input field
WebElement userNameInput = driver.findElement(By.id("input_user_name"));
wait.until(ExpectedConditions.visibilityOf(userNameInput));
// type user-name in input field
userNameInput.clear();
userNameInput.sendKeys("android1@example.com");
driver.findElement(By.name("password")).sendKeys("password");
// submit login form
driver.findElement(By.name("login")).click();
WebElement confirmButton = driver.findElement(By.name("grant"));
wait.until(ExpectedConditions.visibilityOf(confirmButton));
confirmButton.click();
// we are now logged in app and we proceed with native app
driver.switchTo().window("NATIVE_APP");
// find button with label "button1".
driver.findElement(By.name("button1"));
}
@After
public void tearDown() {
driver.quit();
}
}
|
|
Backward compatibility
Appium supports all Android API levels but there is one limitation. As it uses uiatomator for tests running on API>=17, for older APIs you need to run tests using
Selendroid.
Selendroid vs Appium
Selendroid and Appium are very similar:
- both use Selenium WebDriver
- both could be used for native, hybrid and mobile web apps
- both could run tests on emulator or real devices
- both are suitable for Cloud Based Testing
Selendroid, or “Selenium for Android”, is a test automation framework which drives off the UI of Android native and hybrid applications (apps) and the mobile web. As you can see from its name, it could be used only for Android which is not case with Appium (it supports iOS and FirefoxOS, too).
Selendroid has multiple Android target API support (10 to 19) and it has not limitation with WebView testing like Appium which uses uiautomator for API>=17.
UI elements locating is easier in Selendroid. In Selendroid you can find UI element by its id, class, name, xpath, link text, partial link text. Appium, for example, does not support elements locating by id (in layout *.xml file defined as “android:id=@+id/some_id”). It is because uiautomator does not support it for API less than 18. Elements locating by link text and partial link text is also not supported by Appium.
Selendroid has very useful tool called
Selendroid Inspector which simplify UI elements locating. Perhaps Android SDK has
uiautomatorviewer, Selendroid Inspector is more user-friendly.
Limitations
For recognizing UI elements, the Robotium is much more accurate because it lets tests to click on elements by their resource ID that provides a more accurate element identification. In addition to ID, the elements can be recognized by the content. Uiautomator has a general accessibility on labels, e.g. text, description… etc. But if there are more elements with the same text, there is need to add index for instance. And, if the UI changes dynamically, it might be a big problem. As uiautomator actually lets a test to click through device and text descriptions, such as “Settings”, can cause issues as there are “Settings” and “Option settings”. For this reason it is much harder to write an universal test in uiautomator.
Basically, you can easily find every View which has defined
“contentDescription” attribute or which extends TextView class. If you have custom View, which does not extend TextView, it will be very hard to find it by test. Of course, there is an option to find view by xpath, but it is not trivial.
At a time when I was researching Appium I was not able to test screen orientation change or connectivity change. Also I did not find a way how to confirm AlertDialog in my tests. There were some proposals to use javascript methods for this but it did not work for me. Last thing which I was not able to test are
auto-complete text suggestions. I did not find how to select one of suggestions.
Limited support for gestures: If your app uses only simple gestures, like tap, you could be fine. Appium allows you to write javascript wrappers to support different gestures. But you will probably spend a lot of time to write support for them.
Cloud Based Testing
Cloud testing is a form of software testing in which web applications use cloud computing environments (a “cloud”) to simulate real-world user traffic. It is interesting to me because Appium is suitable to run tests in cloud. For example,
SauceLabs or
testdroid provides services to run Appium tests on real devices or simulators. Of course, you need to pay for this but it has a lot advantages compared to tests run on local machine or jenkins. Simulators in Cloud are much faster than emulators running locally.
Conclusion
Appium is still young and I think that it need to grow more to cover all testing requirements and I hope it will. I like idea, especially that I can communicate with other apps on my phone while running the test for certain app which is limitation of Robotium, for example. Cloud Based Testing has a lot of advantages. For example, our tests often fail on Jenkins because it runs tests on emulators which are slow and unpredictable especially when you have wait-for-view conditions in your tests.
