How to explain to management that test automation is not a magic bullet.
Here at Tricentis, we talk a lot about common test barriers to automation. But as one commenter pointed out, we missed one: addressing misconceptions with management about what test automation is, what it should do and how it will affect the QA team.
According to Jim Hazen, who has worked with test automation tools since the early 1990s, “Misconceptions, myths and false expectations by management set up automation implementations for failure.”
Hazen, who has helped design test automation strategies for companies like Jeppesen and Nelnet, says he has run into the issue many times, at a variety of organizations. The lesson he’s learned? “It all comes back to setting proper expectations with management. They hear about automation and think they can put in a tool that can solve all their problems.”
Of course, that’s not exactly the case. Hazen says it’s critical to begin setting expectations as soon as test automation becomes a topic of conversation. “What I usually do is sit down with them and get an idea of their level of understanding, then fill in the gaps. It’s more of a consultative interaction.”
Managers need to understand from the outset that test automation is a tool or a set of tools. And like any other tool, test automation is something that augments testers’ existing efforts and helps them complete tasks with greater efficiency.
“Test automation allows you to look at your testing from a different viewpoint: ‘How can I divide up the problem?'” Hazen explains. And that often leads to creative solutions that improve the entire testing process. But to get that benefit requires an experienced QA professional who can develop a test automation strategy and determine where test automation can help the most.
“You’ve gotta sit down and look at what you want to automate, why you are going to automate, and what the results are,” Hazen says. Testers should point out that this process takes time – and that it will also take time to learn the tools. Test automation is not a plug-and-play, set-it-and-forget it, solution.
Another common fallacy Hazen has encountered is too much focus on ROI or cost savings. Instead, testers should focus on communicating the benefits of test automation in terms of efficiency gains. “What you’re really doing by repeatedly running automated checks against the system is asking, ‘Is this thing stable, and is it the same as what I did before?'” Automated tests can alert testers to where problems might be, so they can focus their attention on those areas first.
If tests can be automated and run on their own, some managers might question how that changes the tester’s role. In these cases, a quick explanation of the incredibly diverse test automation tool landscape should do the trick. The tooling is complex, and selecting the right mix of tools requires careful consideration.
Once tools have been selected and the automated test scripts are written, testers should run through by hand first to make sure the script is testing what it’s supposed to test and yielding the desired information. All of this requires an experienced tester who is familiar with the product and can design a test automation strategy that meets business goals. Hazen recommends putting together a proof of concept to see if the tool performs as expected. Outlining these specifics can also help illustrate to management exactly how automated tests fit into a comprehensive testing strategy – and the QA team’s critical role in defining it.
This sentiment was echoed by several respondents to a recent survey we conducted about test automation trends. One respondent wrote that their biggest challenge was “Improving the mindset that automating scripts will require less resources than manual testing, [and] therefore yields a decrease in QA resources – when it does not.”
Another respondent wrote that they struggle with re-educating people who assume test automation is free or has minimal cost.
Hazen says he has encountered managers who want to use open source tools to save money. Open source tools are great for a variety of reasons, but there are hidden costs associated with using them that managers may not consider. “To implement, maintain and use an open-source tool isn’t free,” Hazen says. “Take Selenium for example. Selenium is an API for a web browser and you have to wrap it with a programming language, integrate it with JUnit and then get another library that’s reporting. You’ve got to compile all this and set it up and maintain it, and that takes time.”
Hazen recommends having a conversation about what is required to maintain a diverse test automation toolset in advance – and, if possible, setting aside a dedicated maintenance budget.
It might be uncomfortable, but having a conversation with management about what test automation can and can’t do for the organization can help testers uncover where test automation misconceptions are so they can address them in advance. It’s also an opportunity to establish some shared goals and to ensure your team has the necessary resources for successfully incorporating test automation into your QA strategy.