Mobile Testing

Exploring the Unsolved Challenges in Mobile Test Automation [Webinar Recap]

We’ve come to expect a lot of mobile apps. From secure banking to gaming, we want apps to deliver impeccable experiences, and the slightest hiccup in performance will prompt us to delete the app from our phones. Enterprises find themselves heavily dependent on mobile apps, yet mobile testing is not always integrated into broader testing strategies.

This might be because mobile testing gets complex… fast. New programming frameworks, app versions, input types and device types are just a few of the variables that mobile testers need to consider. This is what we delved into with Pradeep Soundararajan, Founder of Moolya and AppAchhi, in the webinar “Exploring the Unsolved Challenges in Mobile Test Automation”.

There are multiple ways to to approach unsolved challenges. This came through in the webinar, as we tackled some of the tough questions that mobile testers wrestle with:

  • How much do I really need to test?
    • Point (Pradeep): No project or team has the resources to test every scenario, so many try to reach 80% coverage as a rule of thumb. However, on a project with a financial app, the 20% which wasn’t tested included a particular phone the CXO was using, damaging their impression of the effectiveness of testing. Making sure the revenue-generating users are covered is important, but so is understanding who your stakeholders are when finding ‘the right 80%’.
    • Counterpoint (Christian): Testers should evaluate their test suites against the level of business risk coverage they provide, working with product owners and business analysts to define how to assign risk weightings. This gives testers a way to quantify the value each additional test brings to the table.
  • Cloud execution or on-premise?
    • Point: A past client found that cloud infrastructure providers cost them double what they would have spent with a local device farm, including a 2-person team to maintain the lab. So what are the circumstances under which switching to cloud execution makes sense?
    • Counterpoint: Those who are focusing on testing functional scenarios and business logic will find that local test environments meet their needs. The need for a cloud solution comes into play as tests need to be run across multiple frameworks, network strengths and so on.
  • How can AI be introduced to mobile testing?
    • Point: AI is already being used for visual tests on mobile applications, and may be applied next in performance testing, device configuration, and drawing context from analytics to execute actions. Functional testing was considered an area beyond AI (so far) since it requires a lot of interaction and troubleshooting from testers.
    • Counterpoint: Many interesting ideas have emerged applying AI to testing, but current iterations still need testers to sort through a lot of the output. We have not yet seen an AI solution which reduces overall effort for the tester and allows them to focus on the more important parts of automation.

Watch the full webinar on-demand to hear both sides of these and other hot mobile testing issues, then decide which approaches make the most sense for you and your team.