continuous testing what next

Test automation

The Evolution of Test Automation: Survey Results Are In!

Test automation adoption will grow significantly in the next year, with large enterprises leading the charge, according to a report released by Tricentis and TechWell today. The report, The Evolution of Test Automation, summarizes findings from a survey of testers and QA leaders at midsize and large enterprises across the U.S.

Read the Report

Findings indicate that test automation is still an emerging practice, with most organizations (76%) reporting that they automate less than half of all test cases today. But they’ve got big plans for the future: Almost half (44%) of respondents expect to automate 50% or more of all testing within the next year.

According to one respondent, We haven’t automated enough to see significant improvements as of yet, but have high expectations as we roll out more widespread automation.

So why the sudden push for more test automation? There are clues in the benefits that survey respondents reported. Organizations that automate 50% or more of their tests report benefits including faster testing cycles (88%), improved test coverage (71%) and the ability to catch bugs earlier (68%).

In a press release announcing the availability of the report, Tricentis’s VP of Enterprise Evangelism Elise Carmichael shared her perspective on the findings. The transition to agile and DevOps practices has prompted a major shift in testing strategies that streamlines QA and development, shifts testing left and supplements manual testing with automation.

In terms of company size, respondents at large enterprises said they plan to expand their test automation strategies the most. Only 17% of large enterprises have adopted test automation, but 57% report that they plan to have at least half of all test cases automated within the next year.

This rapid expansion will require enterprises to dedicate significant resources to test automation over the next 12 months, demonstrating the strategic position QA and testing have assumed in the enterprise, as organizations across industries focus on getting higher quality software to market faster.

Testers and developers share test automation responsibilities

Nearly a third (27%) of organizations report that QA and developers share testing responsibilities, compared with 63% where QA is responsible for all testing. Among organizations where QA and developers share testing responsibilities, average automation levels are higher, at 31% to 40%, demonstrating a positive correlation between test automation maturity and cross-team collaboration.

This finding highlights the need for test management software that integrates QA into agile workflows and allows testers and developers to collaborate across tools in real-time. Respondents report using a wide variety of open-source test automation tools, with 83% using at least some open-source tools.

Behavior-driven development complements test automation

A third of the organizations surveyed are using behavior-driven development (BDD) in conjunction with test automation. These organizations also have a higher automation rate, at 41% to 50%, compared with a mean of 31% to 40% in the overall sample. Carmichael said the finding is supported by the growing demand among Tricentis’s customers for a solution for testing in BDD environments.

We believe that because BDD better aligns testers with business goals, we will see an uptick in this approach as enterprises continue to invest in quality as a strategic driver of business success, Carmichael said.

Organizations performing test automation report significant benefits, yet respondents tend to agree that manual experimentation and exploratory testing are essential to success. By eliminating rote tasks, automation allows them to spend their time more strategically – both planning and executing automation strategies and performing the exploratory testing that cannot be replaced by automation.

According to one respondent, We don’t use automation to ‘reduce’ manual testing; we use it to augment it and allow our human testers to focus on new functionality and high-value testing work.

Another summed it up this way: Test automation improves your ability to determine the stability of code build and readiness for deeper testing.

The report also covers the most popular test automation tools and frameworks and the most common benefits and challenges organizations face when implementing or expanding test automation.

Download the report here.