Global Leader in Athletic Apparel

How a global leader in athletic apparel accelerates SAP S/4HANA migration with Tricentis Tosca’s end-to-end test automation

Company overview

Founded in 1949, this global leader in the sporting goods industry designs and distributes athletic and lifestyle products including apparel, footwear, and accessories. As one of the top global athletic apparel organizations, maintaining a firm grasp on their many complex end-to-end business processes is a top priority to keep their eCommerce presence reliable and performant, as with every global retail organization. As such, their QA strategy for such a massive, complex business application ecosystem required intelligent test automation to support this scale.

The organization’s teams were performing end-to-end testing manually, which became too time-consuming and required a lot of human effort, making it difficult to scale. As the organization was set to undergo a (still ongoing) SAP transformation and shift from a legacy, on-premises SAP ECC system to SAP S/4HANA running on cloud infrastructure, it was time to explore more efficient ways to test. The migration was a business-critical project that would enable adidas to digitize core business processes, become a more data-driven business, support new business models such as direct-to-consumer, and scale to meet the needs of a rapidly growing e-commerce business.

The organization’s IT leaders put together an SAP transformation project plan and team and began exploring automated testing solutions that could support a faster, more modern test strategy both during and after the migration.

  • Arrow Icon
    Industry: Consumer products
  • Arrow Icon
    Organization size: 50,000+
  • Arrow Icon
    Location: Europe
  • Arrow IconProducts:

Challenges

  • Lack of scalability with manual test efforts
  • Time-consuming for product team members
  • Regression happening within the testing phase itself with insufficient time to check for it mid test sprint

Finding an end-to-end test automation tool with comprehensive SAP support

The organization’s platform engineering test automation team began to investigate the value add for test automation for end-to-end testing across the organization’s various products, business processes, and technologies and uniting them with one test automation tool.

The team considered UiPath and Micro Focus UFT as well as a few open-source tools like Selenium and Playwright for this SAP migration project. They supplement their testing strategy with these other integrated testing tools as a holistic test automation framework and pipeline for their overall IT ecosystem, not just SAP. While the organization still uses Selenium and Playwright outside of SAP testing, they ultimately selected Tricentis Tosca for their critical SAP migration and general SAP ecosystem testing.

The org’s lead SAP Test Automation Architect said his team chose Tosca based on “a mix of developer preferences and the automation capabilities of the tool versus the requirements we have and what we are hoping to achieve.” According to this lead architect, “The recommendation of SAP and what it entails was one of the biggest factors why we decided on Tosca.”

The org’s testing team also liked the Tricentis and SAP support for Tosca because it means the tool is always up-to-date with SAP changes and is compatible with any SAP technology. This co-innovation approach ensures that the organization and other customers get automated testing out-of-the-box for all new SAP solutions.

Tosca POC proves 288X faster test results over manual

The org’s test automation team, in collaboration with the global test management teams and a select few key stakeholders, started with a proof of concept (POC) of Tosca to demonstrate to its stakeholders how the tool’s test automation could add value to their end-to-end testing flows. The team looked at a few of the end-to-end test scenarios and identified one critical scenario for the POC that could be improved with automation.

The team used a scenario that consisted of seven test cases. It created a roadmap to compare manual to Tosca automated test cycles based on the pain points it heard from its testers when they do end-to-end testing and collaborate with other teams. The main issues were around logistics, time, and the frequency of execution.

“We were trying to understand where the team could add value, what kind of value, what sort of limitations might be in place, and what strategies we could apply to get the most out of automated testing,” said the SAP Test Automation Architect.

The POC compared three test scenarios – manual execution isolated, manual execution end-to-end, and Tosca automated end-to-end – to see the differences in speed, ease, and confidence between manual versus automated testing of SAP S/4HANA and SAP Ariba workflows.

The manually executed isolated test scenario involved:

  • 2 people
  • 1 hour to execute
  • 2 human hours of effort to execute

But when the same seven test cases were tested with a manually executed end-to-end test scenario, the results showed increases in difficulty and time:

  • 5 people
  • 2 to 3 hours to execute
  • 13 human hours of effort to execute

In comparison, the results of the Tosca automated end-to-end test scenario showed:

  • 1 person
  • 25 minutes to execute
  • 10 minutes of effort to execute

The org’s internal analysis of manual versus automation shows that Tosca produced 288x faster test results and 78x less effort to execute. These numbers include the lead time from planning to the execution of results and include the massive time and effort savings from eliminating much manual coordination of multiple teams for each testing sprint.

The qualitative benefits of test automation

Under their previous test configuration, the team was forced to repeatedly extend the testing phase due to the time and effort required to manually execute these test cases. Now with test automation, they could achieve overall logistic simplification with quicker executions, less effort and costs, and earlier defect detection and resolution.

Some of the benefits achieved included:

  • Time savings: Automated scenarios execute faster, allowing for shorter testing phases.
  • Effort and cost reduction: Automating manual tasks reduces manual effort resulting in less effort and costs to test.
  • Earlier defect detection: Since efforts and costs to test are reduced, the tests can be done more often, and potential defects are found earlier.
  • Regression testing: The team is now able to check for regression after corrections are applied to the defects found during the testing phase.
  • Simplification of testing across different test data combinations: The team can check additional nuances within their test processes and can simply reuse the same automations and add the additional test data variants as appropriate.

The org also analyzed the qualitative benefits of applying automation for larger versus smaller end-to-end test scenarios and found these insights:

  • The scope included six end-to-end test scenarios, 151 test cases, and the test phase lasted seven weeks.
  • The smallest end-to-end scenario had seven test cases and the biggest had 58 test cases – the increased size increases the time and effort exponentially.
  • For resources, the testing team estimated it took 1,081 man-hours (the equivalent of 135 days) to execute 100% of the end-to-end scenarios.

“We looked at both smallish and bigger end-to-end scenarios because the approach to test management is different. We found that bigger scenarios require a different approach and exponentially more resources, time, people, and effort,” said the SAP Test Automation Architect.

After the successful POC, the team started offering E2E automation as an additional service to their stakeholders for further support, and the team is now in the early stages of its SAP S/4HANA migration, already seeing the benefits of test automation.

“One strength we saw in Tosca aside from the SAP partnership and the trust that it provides, is the tool really came into its own with its versatility in the systems it supports,” said the SAP Test Automation Architect. “We were able to help a lot of different teams, by supporting SAP Fiori, SAP Ariba, SAP GUI with the same tool, that other tools could not support”, as well as API testing and non-SAP tools, and web testing through multiple different UIs. We saw benefits in that application of test automation that we would not see with other tools right out-of-the-box.”

With plans for future total end-to-end test automation in place, the team can confidently move forward with the significant testing required for SAP S/4HANA migration. As a next step, the team is working on a plan to implement continuous performance testing for SAP with Tricentis NeoLoad.