In today’s environment, many companies are under pressure to automate more and spend less. If your goal is to reduce testing costs, investing in new technology may be counterintuitive. But if you are one of the majority of enterprises still doing 80% of your testing manually, this kind of investment can yield a rapid return.
Scaling test automation can significantly reduce testing costs — one Tricentis customer reported savings of $500,000 each month when compared with the cost of testing manually. AI-powered impact analysis can reduce testing costs even further by pinpointing the specific areas of the application that are impacted by a given release, then identifying the test cases required for testing those areas. Together, these technologies enable a focused, tailored, intelligent approach to testing that achieves both broader coverage and significantly accelerated test cycles. The following guide shares how you can put these technologies into action to reduce both testing costs and timelines — allowing your organization to deliver business value faster and get innovations to the market earlier.
Traditional Testing Methods and the Cost-to-Value Transition
Traditional development models pushed testing to the end of the development cycle, where bugs are expensive to fix and often cause release delays. Yet even for organizations who have modernized development and operations processes, testing still stands out as a bottleneck. A recent GitLab survey revealed that testing represents the most significant delay in the software delivery pipeline, even on DevOps teams.
Recent research from Tricentis and the World Quality Report shed light on this finding. Both reports show that most testing still occurs manually. According to the research, the majority of organizations automate 20% or less of their total testing effort.
That means that for the average organization, 80% or more of testing is still performed manually. And this effort is not without cost: Testing still represents nearly 30% of IT budgets, on average, according to the 2019-20 World Quality Report.
Testing is often the last element of the software delivery pipeline to modernize, but organizations who de-prioritize it are missing a significant opportunity to both save the business significant amounts of money and add significant value, by improving both quality and the ability to adapt to customer needs.
3 Steps to Reduce Testing Costs
Based on many years of R&D, Tricentis has developed the industry’s No. 1 test automation platform, which has enabled many enterprises to rapidly increase automation rates, focus testing efforts and dramatically reduce testing costs. Here are some of the best practices these customers have implemented to transition testing from a necessary cost to a value driver.
1. Automate at Scale
Many organizations struggle to scale test automation because they lack the required coding skills. With these skills in high demand and short supply, resources are often dedicated to the most visible or mission-critical areas – meaning not all parts of the business are covered and overall test automation rates remain low.
Tricentis has developed a model-based approach to test automation which does not require coding skills, enabling enterprises to rapidly scale test automation with the resources they already have. This approach breaks automation down into reusable building blocks, or models, which can be accessed and reused across projects and teams. If most or all of your testing is still performed manually, a scriptless tool offers a way to rapidly scale test automation with your existing resources.
After moving away from manual testing and significantly increasing test coverage, the global payment processor Worldpay calculated that achieving the same level of coverage manually would have cost the company $500,000 each month.
Allianz, one of the largest insurance and asset management providers in the world, recently automated all of their manual test cases with Tricentis Tosca, which increased testing efficiency by 80%.
2. Implement a Risk-Based Testing Approach
A risk-based testing approach catalyzes the benefits of automation. With risk-based testing, testing teams can identify and create the tests that are most important for minimizing business risk – significantly reducing the total number of test cases that need to be created and run before the release can be deemed production ready. Risk-based testing aligns testing activities with business priorities. With this focused approach, you’ll reach optimal risk coverage more quickly and assess release readiness with more confidence.
Tricentis Tosca offers a risk-based testing approach that reduces testing efforts by exposing risks to both the user experience and critical business processes. The resulting test suite is perfectly suited for continuous execution as part of the delivery pipeline, and because it’s built with model-based test automation, it’s easily updated as the application evolves – all without coding or scripting.
The medical device manufacturer Varian reduced testing time by 93% and costs by 35% after transitioning from manual to automated testing for enterprise applications. The company’s complex enterprise app landscape includes SAP, SQL Server, Dell Boomi, Salesforce, ServiceMax and more — each with independent release cycles. Manual testing was unable to support the complex custom work processes being deployed to production every two weeks. With Tricentis, Varian fully automated packaged application testing in just six months, without the need for technical expertise, and significantly reduced total test cases by identifying those that would cover top business risks.
Dolby Laboratories recently automated business process testing across a large SAP footprint and a diverse suite of technologies. Mamatha Mitr, Senior IT Manager, the Head of Strategy for Dolby’s Testing Center of Excellence, said that the organization knew automation would be critical to their success but didn’t know where to start. Her team selected a group of applications that were “financially significant” to the organization and began a 90-day pilot with Tricentis Tosca.
During the pilot alone, Dolby reduced regression test cycle times by 75% and production defects dropped close to zero. A risk-based approach clarified which tests were most critical and saved a significant number of hours for its business system analysts and other functional leads. Today, the organization has automated close to 65% of testing across the identified financially significant applications.
“Now, we can run the entire regression test suite immaterial of where the change is, so the amount of coverage we get is close to 100%. Running test cases that are frequently used and are critical to our business is very important and it would have been close to impossible for us to get anywhere near 100% without test automation software such as Tricentis Tosca,” Mitr said.
3. Understand the Impact of Enterprise Application Upgrades
Rather than running an entire regression test suite for each update to enterprise applications, organizations should work to understand the impact that these updates will have on critical business processes and focus testing on those areas. Traditional impact analysis tools will reveal what areas of the system have changed as a result of the update, but a tool that also identifies the areas that should be tested as a result can achieve much greater efficiencies.
Tricentis LiveCompare’s AI-powered impact analysis focuses the test scope down to the impacted objects within each release. Any testing gaps discovered are automatically added as requirements in Tricentis Tosca so that automated test cases can be created to fill the gaps. By pinpointing the most at-risk areas that need to be tested, LiveCompare reduces testing timelines by 85%, on average, while ensuring 100% risk coverage. Today, LiveCompare is focused on SAP, with plans to extend functionality to Salesforce and other enterprise applications in 2020.
The Coca-Cola Company turned to Tricentis to reduce the overwhelming amount of manual analysis necessary to identify every object and coinciding executable process change associated with SAP updates. SAP is at the heart of The Coca-Cola Company’s functional operations, so the quality of releases gets very high internal visibility. But IT struggled daily to balance project and time management, efficiency and accuracy and risk mitigation.
“It’s a top priority with the CDO and CIO,” said Ray Berry, Application Development Lead for the SAP Platform at Coca-Cola. “There is strong emphasis on quality management and stable releases.” To ensure those needs were met without exhausting IT resources, Berry implemented Tricentis LiveCompare.
“LiveCompare ensures we’re testing the right things and spending resources in the right places. When developing a regression test plan, the data received from LiveCompare allows us to more accurately tailor each plan to an individual release. Instead of testing everything we test only what’s impacted. That has translated into a direct resource savings of 30-40%,” Berry said.