Events
Featured
Get updates on Accelerate 2022

We're back in person. Join us in September for Accelerate in Santa Clara, California.

Sign up for updates
Transformation
Featured
Your transformation toolkit

Advance your enterprise testing strategy with our transformation toolkit.

Learn more
image

Guides & Insights

5 best practices for mobile testing success

Do you remember your first mobile phone?

It may have been an old-school flip phone or one of those that had the slide-out QWERTY keyboard.

The world’s first modern smartphone was launched in 1994 in the United States and was called IBM Simon. It is estimated that around 30,000 people owned a smartphone in 1994. Now about half of the world’s population owns a smartphone.

Your smartphone probably isn’t the only device you have charging at your home or office. Mobile devices now encompass tablets, e-readers, and wearables such as smart watches. Mobile has quickly outpaced desktop as the primary source of web traffic worldwide. Mobile devices generate nearly 50% of global website traffic, with retail, media, business, and finance having the highest growth rate.

The sharp increase in mobile usage means consumers are demanding a reliable experience with the applications across their many devices. To deliver high-quality mobile applications, testing teams must contend with near-endless scenarios in a very fragmented environment – from new OS to old OS versions and modern devices to ones that look a bit ancient. But there are solutions to mobile testing’s challenges that can make the testing process smooth and successful. And those teams who get it right stand to make significant gains in customer reach, revenue, and reviews.

Why mobile testing is critical to an application’s success

Poor reviews on the App Store or Google Play are the equivalent of movie critics panning a new release. Consumers are particular, and research shows that 50% of mobile users won’t consider using apps with a 3-star rating, and 85% of users will skip apps with 2-star ratings.

According to Apple, on average, over 40% of app rejections are for its Guideline 2.1 – Performance: App Completeness. This guideline helps make certain that an app is ready for use. Common reasons for rejection are:

  • Crashes and bugs
  • Broken links
  • Substandard user interface
  • Placeholder content

All these reasons could be mitigated with sufficient mobile app testing. In fact, the top eight iOS shopping apps are tested against a matrix of 41 unique devices, including various OS versions, manufacturer types, screen sizes, and more, while 24 devices are used for the top 15.

5 best practices for solving mobile testing challenges

Testing a mobile application has similar goals as web app testing – ensuring the application software for quality, functionality, and usability. But that’s where the similarities end. Mobile testing brings its own set of challenges for achieving these goals. Here we take a closer look at some of the common problems mobile testing teams face and the best practices you can use to address them.

Challenge #1: Device sprawl widens the test scope

You have your iPhone, iPad, Apple Watch, and maybe a vintage iPod still lying around. All those devices have various versions, with new ones being released constantly. And that’s just iOS, with 26% of the worldwide market share for mobile operating systems. 

Android is at the top at 73%. And then you have Samsung, KaiOS, and a few others filling in the remaining percentages.  

The point is that the world is full of numerous types of mobile devices, all with their own operating systems, versions, updates, and other distinctive permutations. If mobile testing teams were to test all the available devices, they’d be working 24/7 all year long.

Mobile testing teams need a more straightforward device management approach that works for multiple devices and different mobile operating systems and versions. Using a mobile test automation platform, such as Tosca, cuts down on the manual testing many teams rely upon to test multiple mobile devices.

Tricentis has up leveled its Tosca mobile testing capabilities with its new release Tosca 15.0. Tosca 15.0 was developed to simplify mobile testing and help narrow the test scope. It aims to reduce test cycle times and increase continuous testing support across mobile versions and devices. Tosca 15.0 supports the latest iOS and Android OS versions and devices and has the ability to automate test suites for iOS with a Windows machine. Plus, it supports cross-platform testing for Flutter-based iOS and Android applications, as well as React Native support.

Challenge #2: Frequent changes create broad testing needs across multiple frameworks

Cccchhhh-anges! There are copious amounts of changes that need to be made across multiple frameworks when conducting mobile testing. Depending on the organization and the skill sets of its mobile development testing teams, they may be using one or a combination of frameworks. The most popular framework is Appium and others are Espresso and XCTest.

Making changes to scripts and maintaining standards across current and emerging frameworks is time-consuming and inefficient for testing mobile applications. Teams need a codeless testing solution.

Tricentis Tosca 15.0 lets you create mobile tests in the same way that you create UI and API tests — just drag and drop. No complex setup or scripting is required. You gain the power of Appium without having to deal with Appium’s technical complexity. As an added bonus, the new Tosca Mobile Engine overcomes the infrastructure requirement of having a macOS workstation to automate the testing of iOS devices. Tosca Mobile Engine enhances Appium’s driver and enables you to run iOS test automation on Windows, without the need for macOS or Xcode. Simply connect any iOS device to your Windows machine and start developing and executing automated mobile testing.

Tosca success story

How an automaker speeds toward the ultimate testing experience for mobile apps

A European luxury automaker aimed to optimize process efficiency and analysis by developing custom mobile applications specifically for the maintenance staff at their flagship plant. The manual testing process was disruptive and slow. Testing cycles were consuming 60 business users from 15 departments for two weeks.

“The problem was that existing test automation tools, like Selenium, required development skills… We needed a tool to write test automation without a deep knowledge of some programming language by someone with a little bit of logical understanding of the applications,” explained the company’s Specialist of Control Technology.

Tricentis Tosca was able to meet the automaker’s needs based off the power of automation combined with its model-based test automation approach that would be easy for business testers to adopt. The solution has resulted in:

  • 3X faster delivery of features
  • 70% reduction in testing effort
  • 30 hour test executions down to four hours
Challenge #3: Ensuring localization and internationalization testing aligns with global market needs

We live on a complex planet composed of 195 countries, about 6,500 languages, and an estimated 3,800 cultures. What works for an application in one area of the world will most likely not work in another.

Many teams err in conducting their localization and internationalization testing towards the end of the testing process. It should be at the forefront and align with your organization’s goals and strategies for the app. Who is the user? Where are they predominantly located? What language do they use?

Preparing an app for release on the global market is not a one-size-fits-all. You can’t just translate the language from one to the next or switch it left-to-right, right-to-left, and top-to-bottom. These are some of the significant items that should be tested for localization and internationalization:

  • Time zone
  • Time formats
  • Date formats
  • Keyboard layouts
  • Address formats
  • Telephone formats
  • Weights and measures
  • Currencies

L10n and I18n

On a broader level, localization and internationalization are together referred to as globalization. Localization is often abbreviated as L10n and internationalization abbreviated as I18n. The 10 refers to the number of letters between l and n and the 18 refers to the number of letters between I and n.

Challenge #4: Lack of real devices for CI/CD causes bottlenecks

Let’s get real about real devices. Like trying to find a four-leaf clover, actual devices are a rarity for testing teams. And if you do get your hands on a real device, it may not be functioning properly or have the operating system you need.

This limited availability of real devices hinders teams’ ability to conduct consistent and continuous test automation for CI/CD. Without powerful and flexible app testing options, testing is slowed down and results in bottlenecks.

Some alternatives, like device farms or purchasing every single device on the market, aren’t feasible or economical options. Testers should try speeding up testing by using real devices, emulators, and simulators. These three methods used in tandem allow you to get the best coverage across mobile devices.

By testing the SUT with emulators and simulators, developers can deploy their apps in the early stages of development and examine the look and feel of the tested mobile app. With Tosca 15.0, Tricentis has enriched the Mobile Engine with support for mobile device emulators and simulators, in addition to real devices, which supports a shift-left approach and can also be used for debugging.

“Organizations no longer can afford to even, let’s say, buy the top 10 most relevant mobile devices. In the old days, if it’s an internal application for employees, they could then say, well, there are only two or three mobile devices that are admitted in our organization. This will definitely not work if we’re talking about an application that’s rolled out to the public because you just don’t know what people are using.”

 Joachim Herschmann, Senior Director Analyst, Gartner
Challenge #5: Realizing the importance of usability testing

Users will continue to use an app only if it proves to be functional, efficient, and there are no performance issues. According to Google’s research, only 9% of users will stay on a mobile site or app if it doesn’t satisfy their needs. In fact, 66% of consumers will take actions that have some negative impact, such as immediately going to another company’s mobile site or app for what they need (40%) or be less likely to purchase products from the company in the future (28%).

These red flags all relate back to the need for mobile usability testing. Design and testing teams are often consumed with other areas of mobile testing they perceive as more crucial and pay less attention to usability testing. Apps have to be able to adapt to fulfill users’ needs, from text sizes and ergonomics to fast loading times and power usage.

Teams should test how real customers interact with the mobile app to remove issues before it hits the market. One area testers should focus on is screen size. Mobile devices have small screens that need to fit a lot of information while maintaining a clear and concise interface.

Testers should observe how the customer interacts with the app’s elements. Are the app’s frequently used sections easily accessible? Does the user hold the phone close or far away to view the text and graphics? Is the user more likely to hold the mobile device vertically or horizontally? All of these questions will give you helpful insights into your app’s ability to win over customers.

End-user satisfaction and a good customer experience (CX) were rated highly by two-thirds of respondents (67%).

CX validation and usability testing received a high rating by most respondents (51%) as a factor on which focus is needed in the post-COVID world.

Source: World Quality Report, 2021-22

Eliminate mobile testing constraints

Although these challenges may be unique to mobile testing, they still resonate with the wider goal of testing teams: to deliver the best quality product to the end user. At their foundation, the best practices we have shared all aim to accelerate releases while eliminating errors and reducing costs.

Tricentis Tosca easily integrates mobile testing into your broader testing strategy. With the powerful Tricentis model-based test automation, teams can rapidly create mobile tests and use them as building blocks for end-to-end test scenarios on native, hybrid apps, or mobile websites on iOS and Android devices.

A single test case definition can be used across real devices, simulators, and emulators. This reuse significantly streamlines test creation, maintenance, and execution so you can rapidly scale your mobile testing efforts.

Continue Reading

Please register for access