Do you remember your first mobile phone?
It may have been an old-school flip phone or one of those that had the slide-out QWERTY keyboard.
The world’s first modern smartphone was launched in 1994 in the United States and was called IBM Simon. It is estimated that around 30,000 people owned a smartphone in 1994. Now about half of the world’s population owns a smartphone.
Your smartphone probably isn’t the only device you have charging at your home or office. Mobile devices now encompass tablets, e-readers, and wearables such as smart watches. Mobile has quickly outpaced desktop as the primary source of web traffic worldwide. Mobile devices generate nearly 50% of global website traffic, with retail, media, business, and finance having the highest growth rate.
The sharp increase in mobile usage means consumers are demanding a reliable experience with the applications across their many devices. To deliver high-quality mobile applications, testing teams must contend with near-endless scenarios in a very fragmented environment – from new OS to old OS versions and modern devices to ones that look a bit ancient. But there are solutions to mobile testing’s challenges that can make the testing process smooth and successful. And those teams who get it right stand to make significant gains in customer reach, revenue, and reviews.
Why mobile testing is critical to an application’s success
Poor reviews on the App Store or Google Play are the equivalent of movie critics panning a new release. Consumers are particular, and research shows that 50% of mobile users won’t consider using apps with a 3-star rating, and 85% of users will skip apps with 2-star ratings.
According to Apple, on average, over 40% of app rejections are for its Guideline 2.1 – Performance: App Completeness. This guideline helps make certain that an app is ready for use. Common reasons for rejection are:
- Crashes and bugs
- Broken links
- Substandard user interface
- Placeholder content
All these reasons could be mitigated with sufficient mobile app testing. In fact, the top eight iOS shopping apps are tested against a matrix of 41 unique devices, including various OS versions, manufacturer types, screen sizes, and more, while 24 devices are used for the top 15.
5 best practices for solving mobile testing challenges
Testing a mobile application has similar goals as web app testing – ensuring the application software for quality, functionality, and usability. But that’s where the similarities end. Mobile testing brings its own set of challenges for achieving these goals. Here we take a closer look at some of the common problems mobile testing teams face and the best practices you can use to address them.
Challenge #1: Device sprawl widens the test scope
You have your iPhone, iPad, Apple Watch, and maybe a vintage iPod still lying around. All those devices have various versions, with new ones being released constantly. And that’s just iOS, with 26% of the worldwide market share for mobile operating systems.
Android is at the top at 73%. And then you have Samsung, KaiOS, and a few others filling in the remaining percentages.
The point is that the world is full of numerous types of mobile devices, all with their own operating systems, versions, updates, and other distinctive permutations. If mobile testing teams were to test all the available devices, they’d be working 24/7 all year long.
Mobile testing teams need a more straightforward device management approach that works for multiple devices and different mobile operating systems and versions. Using a mobile test automation platform, such as Tosca, cuts down on the manual testing many teams rely upon to test multiple mobile devices.
Tricentis has up leveled its Tosca mobile testing capabilities with its new release Tosca 15.0. Tosca 15.0 was developed to simplify mobile testing and help narrow the test scope. It aims to reduce test cycle times and increase continuous testing support across mobile versions and devices. Tosca 15.0 supports the latest iOS and Android OS versions and devices and has the ability to automate test suites for iOS with a Windows machine. Plus, it supports cross-platform testing for Flutter-based iOS and Android applications, as well as React Native support.
Challenge #2: Frequent changes create broad testing needs across multiple frameworks
Cccchhhh-anges! There are copious amounts of changes that need to be made across multiple frameworks when conducting mobile testing. Depending on the organization and the skill sets of its mobile development testing teams, they may be using one or a combination of frameworks. The most popular framework is Appium and others are Espresso and XCTest.
Making changes to scripts and maintaining standards across current and emerging frameworks is time-consuming and inefficient for testing mobile applications. Teams need a codeless testing solution.
Tricentis Tosca 15.0 lets you create mobile tests in the same way that you create UI and API tests — just drag and drop. No complex setup or scripting is required. You gain the power of Appium without having to deal with Appium’s technical complexity. As an added bonus, the new Tosca Mobile Engine overcomes the infrastructure requirement of having a macOS workstation to automate the testing of iOS devices. Tosca Mobile Engine enhances Appium’s driver and enables you to run iOS test automation on Windows, without the need for macOS or Xcode. Simply connect any iOS device to your Windows machine and start developing and executing automated mobile testing.
Challenge #3: Ensuring localization and internationalization testing aligns with global market needs
We live on a complex planet composed of 195 countries, about 6,500 languages, and an estimated 3,800 cultures. What works for an application in one area of the world will most likely not work in another.
Many teams err in conducting their localization and internationalization testing towards the end of the testing process. It should be at the forefront and align with your organization’s goals and strategies for the app. Who is the user? Where are they predominantly located? What language do they use?
Preparing an app for release on the global market is not a one-size-fits-all. You can’t just translate the language from one to the next or switch it left-to-right, right-to-left, and top-to-bottom. These are some of the significant items that should be tested for localization and internationalization:
- Time zone
- Time formats
- Date formats
- Keyboard layouts
- Address formats
- Telephone formats
- Weights and measures
On a broader level, localization and internationalization are together referred to as globalization. Localization is often abbreviated as L10n and internationalization abbreviated as I18n. The 10 refers to the number of letters between l and n and the 18 refers to the number of letters between I and n.
Challenge #4: Lack of real devices for CI/CD causes bottlenecks
Let’s get real about real devices. Like trying to find a four-leaf clover, actual devices are a rarity for testing teams. And if you do get your hands on a real device, it may not be functioning properly or have the operating system you need.
This limited availability of real devices hinders teams’ ability to conduct consistent and continuous test automation for CI/CD. Without powerful and flexible app testing options, testing is slowed down and results in bottlenecks.
Some alternatives, like device farms or purchasing every single device on the market, aren’t feasible or economical options. Testers should try speeding up testing by using real devices, emulators, and simulators. These three methods used in tandem allow you to get the best coverage across mobile devices.
By testing the SUT with emulators and simulators, developers can deploy their apps in the early stages of development and examine the look and feel of the tested mobile app. With Tosca 15.0, Tricentis has enriched the Mobile Engine with support for mobile device emulators and simulators, in addition to real devices, which supports a shift-left approach and can also be used for debugging.
Challenge #5: Realizing the importance of usability testing
Users will continue to use an app only if it proves to be functional, efficient, and there are no performance issues. According to Google’s research, only 9% of users will stay on a mobile site or app if it doesn’t satisfy their needs. In fact, 66% of consumers will take actions that have some negative impact, such as immediately going to another company’s mobile site or app for what they need (40%) or be less likely to purchase products from the company in the future (28%).
These red flags all relate back to the need for mobile usability testing. Design and testing teams are often consumed with other areas of mobile testing they perceive as more crucial and pay less attention to usability testing. Apps have to be able to adapt to fulfill users’ needs, from text sizes and ergonomics to fast loading times and power usage.
Teams should test how real customers interact with the mobile app to remove issues before it hits the market. One area testers should focus on is screen size. Mobile devices have small screens that need to fit a lot of information while maintaining a clear and concise interface.
Testers should observe how the customer interacts with the app’s elements. Are the app’s frequently used sections easily accessible? Does the user hold the phone close or far away to view the text and graphics? Is the user more likely to hold the mobile device vertically or horizontally? All of these questions will give you helpful insights into your app’s ability to win over customers.
Source: World Quality Report, 2021-22
Eliminate mobile testing constraints
Although these challenges may be unique to mobile testing, they still resonate with the wider goal of testing teams: to deliver the best quality product to the end user. At their foundation, the best practices we have shared all aim to accelerate releases while eliminating errors and reducing costs.
Tricentis Tosca easily integrates mobile testing into your broader testing strategy. With the powerful Tricentis model-based test automation, teams can rapidly create mobile tests and use them as building blocks for end-to-end test scenarios on native, hybrid apps, or mobile websites on iOS and Android devices.
A single test case definition can be used across real devices, simulators, and emulators. This reuse significantly streamlines test creation, maintenance, and execution so you can rapidly scale your mobile testing efforts.