How to modernize SAP testing to reduce risk, increase release speed, and accelerate innovation
A modern SAP environment is critical to the success of any digital business. In fact, updating SAP systems or migrating to SAP S/4HANA is often the cornerstone of an enterprise digital transformation, creating a digital core that connects to applications and ecosystems across the business. Testing ensures that these complex integrations function properly – and that the business will not be interrupted when updates are rolled out.
Because even the smallest software application changes can have a disruptive impact, testing is a critical part of the process. Historically, testing has also been expensive, time-consuming, and complex. But it doesn’t have to be.
A recent survey found that 88% of organizations are planning or have already started a migration to SAP S/4HANA, and expect a significant overhaul of business processes. If you are one of these organizations, you can expect this effort to be both a boon to business and a burden on testing. That’s because neither manual testing nor legacy, script-based tools offer enough efficiency or capability to handle today’s accelerated update cycles.
The following guide outlines strategies that will help you modernize SAP testing, as well as stories of other enterprises who have pulled it off successfully and the impressive results they have achieved.
How manual testing puts SAP projects at risk
Manual software testing for SAP updates and upgrades can take weeks to complete, making it almost impossible for IT teams to be as responsive and innovative as the business requires. If there’s not enough time to complete testing, that means you’re either delaying innovation that is critical to the business or releasing software with unknown defects that could be putting your business at risk.
Software delivery bottlenecks
Traditional testing processes create a bottleneck that impedes business innovation. A recent GitLab survey showed that testing is the most significant source of delay in software delivery, even on DevOps teams. New research from Tricentis shows the same trend in SAP development initiatives, where there is still higher-than-average reliance on manual testing. Because a broad range of processes rely on SAP, from inventory management to sales, purchasing, finance, and accounting, this leads to significant productivity and innovation issues.
Support for end-to-end business processes
Today’s core business processes are much broader and more complex, often crossing multiple technologies, architectures, and deployment models. The average organization now uses more than 900 applications, and a single workflow might touch dozens of these applications via microservices and APIs.
While it was previously possible for software testing teams to test separate business processes on an application-by-application basis, they now frequently have to run multiple parallel tests in order to validate end-to-end process performance. Automating these tests makes it possible to both improve test coverage and reduce testing timelines.
Managing software risks
Traditional manual testing approaches introduce inevitable risks, both through the inconsistency of testing, and by limiting the amount of risk coverage testers are able to achieve. Unlike automated software testing, manual methods can’t support continuous regression testing, where core business processes can be checked every day to quickly pinpoint risks and reduce the risk of business disruption.
Poor IT resource utilization
With manual software testing, pressure often falls on business analysts and users to perform repetitive and time-consuming tasks, taking them away from their usual ongoing responsibilities. Enterprises may also establish hypercare teams made up of experienced SAP project members after each SAP update, again costing their attention and money. Given today’s skilled IT shortage, that’s not an optimal use of business and IT staff’s time, and prevents them from focusing on more strategic, higher value activities.
Acting as a digital transformation drag
Successful digital transformation initiatives are all about innovation, first-mover advantage and getting improved technology into the hands of business users and customers more quickly. Testing delays introduce concerns around uncertainty and risk, particularly when the timeframe to implement major updates is measured in weeks, not months.
Why legacy, script-based tools fall short
To combat these challenges, some enterprises have adopted script-based SAP testing tools. But these legacy test automation tools were built for waterfall development, when teams were managing only a few releases a year. Almost everything has changed since then.
Expensive to build and maintain
Scripts are cumbersome to maintain when developers are actively working on an application. The more frequently the application evolves, the more difficult it becomes to keep scripts in sync. Teams often reach the point where it’s faster to create new tests than to update the existing ones. This cycle quickly leads to an unwieldy test suite that needs to be updated constantly. Exacerbating the maintenance challenge is the fact that scripts are as vulnerable to defects as code – and a defect in the script can cause false positives and interrupt test execution.
Limited test automation capabilities
While legacy tools can theoretically be used to automate regression testing, many users report that regression testing still takes weeks, most commonly due to low test automation rates. In the era of two-week Agile sprints, that kind of timeline is no longer acceptable.
AGL Energy realized they needed to modernize SAP testing when the company launched an initiative to deliver new ideas to production in five days. Achieving this goal would require them to accelerate their quarterly SAP IS-U and SAP CRM backend delivery cycles. The major testing hurdle was their lengthy regression test cycles, which took more than a week. After implementing Tricentis test automation, they were able to automate their end-to-end regression test suite, significantly improve test automation rates, and run more than 95% of their tests on a daily basis.
Today’s complex SAP environments require more than automated regression testing. A complete test automation solution should simplify test automation at the UI and API layers, as well as across end-to-end business processes.
Inadequate risk coverage
Legacy SAP test automation users commonly accumulate huge test suites with unknown risk coverage. Upon review, they discover that only 30% of business risks are covered – and the vast majority of tests are redundant. One Tricentis customer who migrated from a legacy, script-based tool reported spending a year and millions of dollars to automate over 9,000 test scripts. By the time they finished, they had only achieved 8% risk coverage.
Inadequate end-to-end test coverage
When the Linde Group committed to rolling out weekly SAP updates to accelerate their transition to the intelligent enterprise, QA leaders recognized that legacy scripting tools needed an overhaul. At the time, low rates of test automation would make that goal nearly impossible.
The application stack driving Linde’s core business is extremely complex, including 18 highly customized SAP instances and business processes that spanned more than 184 different applications in just one division. Legacy SAP testing processes were equally complex, requiring specialized expertise across Micro Focus UFT, SeeTest, and Selenium.
Linde moved to a scriptless test automation approach, and within six months, they had significantly increased test automation rates for SAP, Salesforce, mobile, web and other technologies involved in end-to-end transactions.
The remedy: Scriptless, model-based test automation
Scriptless test automation allows you to keep up with accelerated release processes by significantly reducing the time, effort, and resources required to test SAP releases.
Model-based test automation enables a high level of test case reuse, which makes it easier to build and maintain comprehensive automation for end-to-end business process tests – with no coding skills required. This reduces testing timelines, as well as the risk of business disruption, by exposing potential errors and defects before they can impact your production environment.
When Varian discovered how much time business users were spending on manual testing for each release, they committed to reducing that burden. The team implemented Tricentis to modernize SAP testing for both enhancement and support pack upgrades and for SAP S/4HANA migration. Today, business users spend 93% less time on testing, and testing costs have shrunk by more than a third.
Arnab Mukherjee, an Enterprise Applications Manager at Varian, explained the critical nature of the modernization effort this way: “With the need to release more frequently across different applications in the landscape, it is imperative that you have an automation framework that lets you accelerate testing.”
Achieve new levels of risk coverage
When paired with AI-powered change impact analysis, this approach can shrink testing timelines even further. Impact analysis identifies both redundancies in the test suite and previously undetected coverage gaps. By highlighting what areas of your applications are most impacted from a given SAP update, impact analysis can help you identify the tests that are most important to run – as well as the ones that aren’t necessary. This allows software teams to prioritize testing according to risk, and to test less while achieving significantly higher risk coverage.
That is what one apparel company discovered when they launched a massive SAP upgrade project using smart impact analysis. To complete the project, the team needed to apply 8 years of SAP Support Packs across 6 landscapes (ECC, BW, SRM, XI, BOBJ, and Enterprise Portal). The ECC updates alone consisted of 62,000 SAP Notes and more than 1.1 million affected objects. By using smart impact analysis to test only the right things, the company was able to complete the entire project in just 9 months and with zero defects.
Use IT resources more effectively
Automation eliminates the burden of manual software testing. This releases potentially tens of thousands of expensive and limited resource hours each year–giving CIOs and their teams much more opportunity to research new IT innovations and bring them to market more quickly.
When Dolby Laboratories relied on manual testing, their business system analysts lacked the time and resources to test each release thoroughly. It was nearly impossible to achieve end-to-end test coverage. With Tricentis test automation, the team reports nearly 100% risk coverage for each release.
According to Mamatha Mitr, Head of Strategy for Dolby’s Testing Center of Excellence, the effort to modernize SAP testing was well worth the effort. “The precise number of hours saved depends on the workstream, but it’s more than 75% of the time saved for business system analysts and other functional leads. Running test cases that are frequently used and are critical to our business is very important and it would have been close to impossible for us to get anywhere near 100% without test automation.”
Unlike other automated testing solutions, Tricentis lets you automate SAP tests without having to wait for a stable SAP environment. Tricentis uses model-based test automation, which is built for use with rapidly changing software. So wherever you are on your SAP journey, you can modernize SAP testing now, start benefiting from automation, and take the benefits with you as you go.
SAP and Tricentis: Partners in your success
Tricentis has a longstanding partnership with SAP to bring the benefits of modern testing technologies and practices to SAP customers. We were the first testing platform to integrate with SAP Solution Manager and we are the first to integrate with SAP Cloud ALM, ensuring support for all generations of SAP software, no matter if you run on-premise, on the cloud, or on a hybrid solution. Through our partnership, we will continue to innovate with SAP so that you can always rely on the Tricentis platform to help you speed adoption of the latest SAP software and solutions, with 10x faster testing speeds, 90% risk reduction, and 50% cost savings.