Two of the most popular drivers for Android app automation are Appium and Calabash. While both have their pros and cons, their major limitations are:
Calabash: Can only drive UI which is part of the application under test; in particular there is no support for testing notifications.
Appium: Cannot call backdoor methods in the application like Calabash (https://developer.xamarin.com/guides/testcloud/calabash/working-with/backdoors/#backdoor_in_Android). Backdoors are very helpful to set the state of application under test.
At Badoo, we use Calabash for automated Android application testing. One of our goal is to make it super easy for developers to run these tests in one command without going through the pain of setting up a test-scripting environment (Installing JDK, Android SDK, Ruby, and praying that nothing else has been added since the documentation was written). The two obvious choices were Vagrant and Docker. Speed of execution was one of the main criteria for us, so we ended up choosing Docker.
Most of the people in our team use Mac laptops and setting up Docker with Calabash-Android on Mac is not straightforward.
In this article which I wrote on Badoo’s tech blog, I have shared the solutions we implemented.
While automated functional tests using appium or calabash acts as safety net, there is something missing.
Often, we find issues like:
“Sign-In button has shifted a bit and user have to scroll down to see it”
“We asked to change background color of screen-X, it got changed for Screen-Y screen as well”
“Why has fonts of this link become so tiny?”
For mobile apps, look and feel of an application is extremely important. With long running projects, any minor
refactoring can change your app’s looks. functional testing tools like calabash Or Appium can not help us in this case.
Nakal gem fills above gap. With this, we can extend any existing mobile automation framework to support automated visual regression testing.
I have been working on automation of android app using calabash for a while. One challenge I see with lots of automated tests is they take long to execute. To solve this problem I created a ruby gem ‘parallel_calabash‘
I tried to keep it simple to use. All you need to do is install this gem or include it in your Gemfile and just run the command as: