Under their previous test configuration, the team was forced to repeatedly extend the testing phase due to the time and effort required to manually execute these test cases. Now with test automation, they could achieve overall logistic simplification with quicker executions, less effort and costs, and earlier defect detection and resolution.
Some of the benefits achieved included:
- Time savings: Automated scenarios execute faster, allowing for shorter testing phases.
- Effort and cost reduction: Automating manual tasks reduces manual effort resulting in less effort and costs to test.
- Earlier defect detection: Since efforts and costs to test are reduced, the tests can be done more often, and potential defects are found earlier.
- Regression testing: The team is now able to check for regression after corrections are applied to the defects found during the testing phase.
- Simplification of testing across different test data combinations: The team can check additional nuances within their test processes and can simply reuse the same automations and add the additional test data variants as appropriate.
The org also analyzed the qualitative benefits of applying automation for larger versus smaller end-to-end test scenarios and found these insights:
- The scope included six end-to-end test scenarios, 151 test cases, and the test phase lasted seven weeks.
- The smallest end-to-end scenario had seven test cases and the biggest had 58 test cases – the increased size increases the time and effort exponentially.
- For resources, the testing team estimated it took 1,081 man-hours (the equivalent of 135 days) to execute 100% of the end-to-end scenarios.
“We looked at both smallish and bigger end-to-end scenarios because the approach to test management is different. We found that bigger scenarios require a different approach and exponentially more resources, time, people, and effort,” said the SAP Test Automation Architect.
After the successful POC, the team started offering E2E automation as an additional service to their stakeholders for further support, and the team is now in the early stages of its SAP S/4HANA migration, already seeing the benefits of test automation.
“One strength we saw in Tosca aside from the SAP partnership and the trust that it provides, is the tool really came into its own with its versatility in the systems it supports,” said the SAP Test Automation Architect. “We were able to help a lot of different teams, by supporting SAP Fiori, SAP Ariba, SAP GUI with the same tool, that other tools could not support”, as well as API testing and non-SAP tools, and web testing through multiple different UIs. We saw benefits in that application of test automation that we would not see with other tools right out-of-the-box.”
With plans for future total end-to-end test automation in place, the team can confidently move forward with the significant testing required for SAP S/4HANA migration. As a next step, the team is working on a plan to implement continuous performance testing for SAP with Tricentis NeoLoad.