One the world’s largest healthcare companies is firmly committed to improving people’s health and well-being through innovation on everything from artificial intelligence to machine learning and virtual reality. In addition to the medical technology that is used in healthcare settings worldwide, this involves constantly updating and re-envisioning innovative internal systems. In all cases, releases require the upmost confidence in quality given the nature of their critical and highly regulated business.
A large part of what allows this organization to achieve its mission are its internal SAP and Informatics systems. These systems touch every part of the organization: sales, materials management, warehouse and plant management, quality management, and finance. Flawless system updates and releases are critical to business and must be delivered at speed.
In the early 2000s, internal efforts were made to improve the process of testing critical systems by introducing test automation. A script-based testing tool (HP UFT) was purchased and deployed. However, it did not meet the expectations of the domain experts and the quality assurance team. The previous tool required significant time to create and maintain test automation and the effort needed outweighed the benefits of automated test execution.
The organization set out to find a tool that could help the organization expand the scope of test automation and help them reach new levels of automation maturity. They recognized that this required a tool that enabled testers and domain experts to create test automation without extensive technical training and keep the automation running without constant updating and disruption.
- SAP ECC systems drive essential business operations; each release must be flawless
- FDA regulatory compliance adds additional scrutiny to each and every system change
- HP UFT test maintenance was time consuming and costly
- Scripted test cases were often not reusable across updates and releases
- Only a limited set of team members had the technical skills required for test automation
- Test execution required considerable time for each sprint
- Company-wide initiative to mature the software testing lifecycle with a “Shift Left” methodology
- Automation not integrated into CI/CD pipeline
After an initial market evaluation, Tricentis Tosca was chosen for its scriptless automation and ease of use for domain experts and automation engineers. In addition to adopting the best-in-class tool, the organization’s leaders wanted to elevate their SAP testing to world-class levels. The client selected Mindtree Consulting as their strategic technology partner to lead the advancing test automation processes from start to finish.
Mindtree is an industry leader in delivering technology services from ideation to execution, with deep infrastructure and application management expertise. Their “born digital” approach takes an agile, collaborative approach to turning IT into a strategic asset to any organization. Mindtree was focused on infusing the right test automation into every layer of testing within the organization to push speed and efficiency.
Regression testing was an early success. Large regression test suites were prioritized for automation according to risk, then implemented with model-based test automation. Test required to be added to the regression suite are now automated in-sprint. Those regression suites are run daily or on-demand to reduce the time, effort, and risk of each core update and new release.
Beyond just regression automation, the customer wanted to stay true to the original goal to extend the coverage of test automation while also creating more mature and robust automation frameworks within the SAP testing group. Automation engineers were upskilled into SDETs and domain experts quickly learned how simple it was to automate sections of their process flow to expedite testing efforts and reduce manual work.
Mindtree offers technology consulting and data transformation consulting by integrating AI to deliver data-driven customer experiences. They are focused on delivering blueprints for digital transformation at the intersection of business, technology, and human empathy. Learn more
Today, automation sits within every aspect of testing and covers 90% of testing for SAP. “We have injected greater levels of automation in every crossroad of SAP testing—from the first line of unit code to integration and functional testing as well as end-to-end testing,” says a representative from Mindtree.
The success from regression automation quickly carried over into other areas of testing, including systems integration testing, and even user acceptance testing. Automation has reduced the required time for execution by 50% and allowed for more frequent execution during the development lifecycle.
Tests are now fully integrated into the CI/CD pipeline to ensure timely feedback once code is deployed. This process has been critical in identifying issues earlier in the testing process, decreasing delays and costs associated with test execution by over 50%.
The speed at which the customer is automating new features is accelerating as well. With the old process, automation could not be introduced until Sprint+2, if at all. Those scripts have shown to be up to 95% operational, meaning the low maintenance of test scripts continues to build ROI from one sprint to the next. Under the guidance of Mindtree, automation occurs in Sprint+1 and is quickly working towards sprint automation for other testing areas to reach Shift Left aspirations set by the organization.
- Achieved 90%+ automation for critical SAP testing
- Decreased test execution time by 40-60%
- Increased testing frequency by 2X to 6X
- 70%+ reusability of automated test cases
- Functional automation integrated into every spring within one quarter
- Automated with Risk-Based Testing for most critical system scenarios
- Fully integrated into CI/CD pipeline
- Functional test automation integrated into every sprint within one quarter
- Automation is now achieved earlier in testing with Sprint+1 within sprint automation on the horizon