Test Automation

Continuous testing

Test Automation Pitfalls for Continuous Testing

Over the years, the concept of test automation has promised more than it can deliver. Now that the move to Agile and DevOps is making Continuous Testing a business imperative, progressing beyond the dismal industry-average test automation rates of 20-30% is no longer optional. If QA testers can’t meet the test automation challenge, they risk being replaced.

For their article on “Avoiding the pitfalls of automated testing,” SD Times approached Tricentis’ Michael Eckhoff and Gerd Weishaar for their take on conquering test automation challenges and why it’s so difficult to achieve high levels of test automation with the available testing tools.

What Are the Top Test Automation Challenges?

Excerpted from “Avoiding the pitfalls of automated testing” by SD Times. See the full article here.

“Michael Eckhoff, software test automation veteran at Tricentis, said one of the biggest pitfalls he sees is companies trying to pick a software automation tool and expecting it to solve all of their problems. What companies really need to do is figure out how to pick the right tool that best meets their needs, and to ensure that the company is ready for it, according to him.

‘You can typically get that first quick win with intuitive test-case design,’ said Eckhoff. ‘Getting to that second phase of really intelligently designing your test portfolio to ensure that you are testing to get the best risk coverage or best business value is where that [best-practice] methodology comes back into play.’

Another risk comes from overconfidence, said Eckhoff. Companies look at their processes, see a large amount of automated test cases, and assume they are in “good shape,” he said. The problem in this case is the tests are not clearly defined, and in many cases, the test definition doesn’t stay consistent with what the test is actually doing, he said.

Companies need to get their teams to buy into the concept of test automation, or organizations will continue to treat agile as the “Wild West,” where teams do whatever they want just to release things fast, and then when it breaks, they do it again, said Eckhoff. He said this way of thinking just doesn’t work for the enterprise, and those organizations need more than just a tool to achieve speed and quality applications; they need a process in place that supports agile development and QA early on.”

Why Testing Automation Tools Fail

When it comes to the test automation tools themselves, the challenges can vary widely. Common obstacles include fragile automation frameworks, a shortage of engineers for test maintenance, unavailable web services, and more. Gerd Weishaar, Tricentis’ VP of Product Management, weighs in:

Excerpted from “Avoiding the pitfalls of automated testing” by SD Times. See the full article here.

“It’s become increasingly difficult for teams to build a test suite that not only covers the risks that are most critical to the business, but also is maintainable and efficient enough to be used for continuous testing of rapidly-evolving applications. Delivering positive user experiences requires continuous testing—and accelerated development cycles mean there’s less time available for designing, creating, updating, and executing tests.

Our solution is designed to help testers rapidly identify and create the tests that are most important for protecting key end-to-end business processes. Based on many years of R & D focused on Model-based Test Automation, we’re able to scan the application under test to help testers understand and define the optimal set of test cases that will cover their organization’s highest priority risks. The resulting test suite is perfectly-suited for continuous execution as part of the delivery pipeline, and it’s easily updated as the application evolves.

All this is accomplished without coding or scripting. Scripted tools lead to long complex test scripts, which get “fixed” in the script if technical challenges arise. At some point these scripts become too many, too complex and end up as maintenance nightmare. This is typically the reason why manual testing is still so popular (80% of test cases) after 20 years of (script based) automated test tools like QTP/UFT.”