Jumbo Supermarkets is the second largest supermarket chain in the Netherlands, and they have been making fans out of customers since 1921. Their mission is focused on delivering a “better, different and fun” shopping experience across almost 700 stores, including a growing segment in Belgium.
Today, Jumbo sets to differentiate themselves through an omnichannel mindset, and aims to be an online market leader. In order to achieve that goal, Jumbo adopted Agile development practices to deliver new features to customers faster. With a growing number of releases each year, manual testing practices were unable to keep up with the pace of change and uphold the quality Jumbo requires. IT leaders at Jumbo knew implementing test automation was one of the first steps to improve the quality of testing and provide greater confidence with each release.
“We need to complete regression testing every release, which is two weeks. But a single regression testing cycle used to require more than two weeks to complete,” says Hanno Koeslag, Development Team Lead.
As a first step, Jumbo test specialists came together to develop a list of criteria needed in an automation tool, to facilitate fast and high-quality testing for each release across the multitude of technologies in the Jumbo landscape. Their list was focused on no-code automation that could easily be facilitated by the existing testers within the Scrum teams, low test maintenance, and the ability to automate multiple technologies (including SAP and non-SAP applications) for complex end-to-end testing.
A search was initiated to determine which tools were available in the marketplace that would meet the stringent criteria set by Jumbo. Once the list of tools was narrowed down, the various automation options were put to the test. A Jumbo-wide decision was made to implement Tricentis Tosca as the test automation tool of choice.
Tricentis partner SUPERP which assisted for the test tool selection was also chosen to help accelerate their test automation efforts with their deep expertise in building and testing powerful business solutions based on the SAP software.
Testers were able to quickly pick up the model-based testing approach with the use of Tricentis’ online Academy training. After three months, tests were already being executed on both SAP ECC and SAP S/4HANA, as well various in-house applications.
– Hanno Koeslag, Development Team Lead at Jumbo
Jumbo currently runs two systems of SAP. The larger, highly customized instance of SAP ECC is at the center of its larger operation in the Netherlands. SAP S/4HANA is a new instance which controls operations in Belgium. Jumbo is in the process of migrating to a single S/4HANA system for both countries. In the meantime, both systems require frequent updating and testing, in parallel. Their Tosca tests run on both ECC and S/4HANA, making testing easier to maintain as the migration progresses.
Test automation has helped the team at Jumbo realize returns quickly. “We saw a 46% of effort reduction in production defect resolution. The actual cost reduction of not having to spend time addressing those issues was more than time we invested in building the scripts over the first three months,” says Koeslag.
Early in their test automation journey, Jumbo discovered an unexpected benefit from their test automation. Previously, the master data team received frequent test data requests from various Scrum teams. This test data had to be generated manually, and required a significant amount of time and effort to produce. With the new test automation scripts, test data was automatically generated that could be used to populate test scenarios later in the process.
“A Tosca article can be created and stored on the test data server, where another process can use that data from the server to perform tests and report back into the server. This process can continue on and pull the same data through the whole supply chain,” says Peter Groothuis, Scrum Master & Test Analyst.