I use mobile devices for most of my online activity now, like checking emails and reading my favorite websites. My desktop PC just isn’t used as much as it was before.
The funny thing is that although I am a techie, I’ve come to learn that I’m far less tolerant of buggy apps on mobile devices than I ever was on any other environment. It may have something to do with apps being very cheap or even free. Checking them out normally costs me nothing but time, and so I immediately delete anything that doesn’t work.
On the other hand, it is important to keep in mind that mobile devices are consumer devices like a TV set rather than a computer – user experience is far more important than all the technology in the background.
For companies going mobile this is quite a challenge – not only do they need to provide (almost) everything for free – they also have to deliver it with very high quality and within a very short time frame.
This immediately leads to a couple of questions:
#1 How can we handle the huge number of devices out there running on so many different mobile OS’s?
Actually, the hardware isn’t the real issue–– it is the version of operating system that you really need to worry about. True, there are more versions out there than usual, but that number is much lower than the number of devices.
Our advice is that you align your testing priorities to the current trend – just look at pages like Android Dashboard or the iOS Usage Graph from time to time to determine what your testing priorities should be. Concentrate on the versions that your users are using.
#2 Do I need to run my tests on physical devices?
The answer is unfortunately yes – emulators are either very slow (as in Android) or restricted to the development environment (iOS). Physical devices are therefore required for efficient testing.
#3 How can I take care of all of this and still stay on schedule?
One answer: test automation. A good test automation tool will give you the time you need to ensure that the quality is what your customers expect. As with other technologies, you have to ensure that you can:
- Use reliable means to identify the controls in your app – this means that you need technical identifiers. Solutions like OCR- or imaged-based recognition can never achieve the same results.
- Create a suitable model of your app – you don’t want spent time on different flavors of edit boxes when all you want is to enter text into a field. Make sure that you don’t need to know so much about the technology underneath. What you really want to avoid is having to recreate your tests anew for every device. In the best case scenario, you can run the tests cross-OS – meaning that you can create test cases for iOS and still run them under Android.
#4 Does this really solve my time-to-market problem?
Good question –before answering it, let’s first consider this:
A newly created mobile client is normally using an already existing infrastructure. That means all different client versions are connecting to the same web service to send and receive data.
A mobile app has very little business functionality – in fact the business rules are implemented in the business layer.
Knowing this, we can concentrate our effort on the web service interface – this layer should take the brunt of testing. The good news is that it is very fast and can thus solve our time-to-market problem.
This way, you can reduce testing costs and effort by 90% percent.
Of course you still need to run some test processes on the mobile device because you want to make sure that the data is transferred correctly to and from the client.
To make use of this, you need a test automation tool which not only supports mobile apps but also supports web service tests.
#5 (Spoiler Alert!)
Think of this one as a bonus answer to all the questions above.
It’s about a secret I wanted to share with you – I know that if I do it on our blog it won’t be a secret anymore – but I am so proud of our upcoming new product that I have to say at least something. We will soon be unveiling a new tool for mobile testing that is a perfect fit for our existing test automation tool suite. We’re calling it Mobile+, and it will be released in late April.
But enough of that for now! Stay tuned for some more thoughts on mobile testing – and maybe some more spoilers on Mobile+.