How different development & delivery methods affect tester scope, status, job satisfaction and more
Tricentis & TechWell surveyed testers and testing leaders across the globe to gather insights about the evolving role of the tester against the backdrop of agile adoption, DevOps transformation and more.
We asked respondents how far along their organizations were with DevOps transformation and used this information to assess how DevOps maturity levels affect testers’ experiences. The standout finding: Among organizations that have completed DevOps transformation, 87% say testing is critical to the success of their DevOps initiatives. These respondents were also more than three times more likely to say their organizations prioritize testing than any other group — and tended to be significantly more advanced in terms of test automation.
Advanced Test Automation is Critical, but Tough to Achieve
Higher test automation levels correlate with DevOps maturity. But achieving a high rate of automation is proving to be more difficult than anticipated. In our survey of a similar audience in 2018, 44% of respondents said they expected to automate more than 50% of their test efforts in the next 12 months. When we asked this year, only 14% had achieved that goal. In fact, most respondents are currently automating less than 20% of their total testing effort.
2018 Expectation
Automate 50%+ of test suite in 2019
2019 Reality
Less than 20% of tests are automated
How Do Testers Spend Their Time?
We asked testers where they spend the most time, then segmented responses according to DevOps maturity level. Respondents reported that testing strategy was more important to the organization than any other task. But when testers ranked how they are actually spending their time, testing strategy only made the top five in organizations that are undergoing or have completed DevOps transformation. In organizations where DevOps isn’t a priority, testers spend the most time on implementation and documentation. Hover over the categories below to explore.
DevOps Complete
DevOps in Progress
DevOps Not Started
Time Spent On Tasks
Implementing Tests
Testing Strategy
Documenting Issues
Designing Tests
Planning Tests
Maintaining Automated Tests
How Does Job Satisfaction Compare?
We wanted to understand how Agile, DevOps and test automation rates affect job satisfaction. Testers in Agile organizations are almost four times more likely to be satisfied in their jobs than testers in waterfall organizations. Across DevOps maturity levels, job satisfaction was lowest in organizations currently undergoing DevOps transformation, and higher at either end of the maturity curve. In mature DevOps organizations, managers are significantly more likely to be satisfied or very with their team’s work. DevOps maturity also correlates with a significant increase in the perceived importance of testing and the level of respect testers receive. Hover over the chart below for percentages.
Asia-Pacific stands out as the region in which testing is prioritized most as part of DevOps transformation, and in which testers are most respected, followed by the Americas. Organizations in Europe and the Middle East had the greatest maturity divide, with both the highest percent of complete DevOps transformation and the highest percent reporting they haven’t started yet. The standout difference in the Americas was the rate at which respondents reported practicing Agile: 85%, compared with 76% in Asia-Pacific and just 30% in Europe and the Middle East. Hover over each region in the map below for more details, and download the report for the full analysis across regions.
devops transformation status
10% Complete
52% In Progress
19% Planned, Not Started
20% Not Planned
Most likely to be practicing Agile development & testing
