How Tricentis and Sofka helped Proteccíon accelerate test automation and reduce costs
Discover how Protección streamlined operations, saved 45% on implementation time, and achieved 65% test automation coverage with Tricentis Tosca.
Headquartered in New Albany, Ohio, Abercrombie & Fitch Co. is a Fortune 1000 specialty fashion retailer with over 730 stores worldwide, and three offshoot brands under management, including Hollister Co., abercrombie kids, and Gilly Hicks.
In the retail sector, survival depends on the ability to adapt quickly to customer needs. Most recently, every retailer has had to grapple with the sudden seismic shift in consumer behavior due to the COVID-related social-distancing mandates and stay-at-home orders. Online shopping has skyrocketed, and customers increasingly demand a fast and reliable digital experience every time, from anywhere, on any device. While these new habits accelerated trends already under way, PwC research shows that these changes are here to stay. Excellent digital performance has become a do-or-die competitive differentiator.
Abercrombie has always been at the forefront of reading the market and reacting to customers’ evolving needs. Its digital footprint includes 40+ eCommerce desktop and mobile sites, a volume that calls for constant focus and effort to maintain performance at every turn. But it was well prepared to meet the digital transformation onslaught — digital helped the company realize its best income performance in over a decade. Digital sales grew by 39% in 2020, to $1.7 billion — more than half of total sales.
Probably nothing challenges eCommerce performance engineers as much as preparing for the holiday shopping season. The stakes are high, and systems are ever more complex. Consider:
Ajay Pedhagandham, Quality Assurance Manager at Abercrombie & Fitch, manages the performance testing team and is responsible for performance test automation. Most important, he’s been at the helm leading the company’s holiday readiness testing team for the past eight years. He notes that any page-response delay, even for a second, can mean the difference between growth and loss. To prepare for the peak traffic loads, his team has to simulate tens of thousands of orders per hour to ensure readiness. He adds, “Our customers are socially active influencers — a bad shopping experience can lead to a firestorm of negative conversation, which can quickly tarnish the brand.”
Given the value that Abercrombie places on performance, it’s no surprise that the challenges Ajay encountered with a legacy performance testing tool revolve around cost and coverage. He explains, “As we were growing and expanding our application delivery practices, which included increased global reach, we knew our performance testing coverage had to support this growth. Our performance testing had to be conducted realistically, so that we could prepare for peak holiday volume. The more our virtual user count needs grew, the more we had to pay. It was becoming cost prohibitive to test under real peak scenarios. Plus, time and again, when we were trying to replicate peak load, we would get a lot of false positives. The reality is, true holiday readiness was not easily achievable.”
Abercrombie & Fitch’s search for a new performance testing solution began in 2009 on the heels of its annual holiday readiness evaluation, which had identified gaps with its legacy solution. The wish list was simple — the company wanted a solution that was both cost-effective and flexible/scalable enough to facilitate realistic performance/load testing out of the box. In addition, the team wanted script maintenance simplicity, something it did not have with its existing performance testing tool.
Abercrombie & Fitch determined that Tricentis NeoLoad offered the best combination of speed and accuracy. Scripting time/test maintenance has been reduced by over 80% — from seven days down to one. Ajay’s team has eliminated the time-consuming complexity of hand-coding tests, and automated script maintenance means they no longer have to rewrite scripts from scratch every time code changes.
“We’re now walking into the war room having achieved our performance testing goals already, and this enables us to focus on backend systems,” says Ajay. This is crucial because while frontend traffic is upwards of 3X more than usual on peak shopping days, the backend load increases by 10X or more. “This extra time allowed us to identify and fix an issue we never would have caught, or had the time to address, had we been focused exclusively on holiday load testing goals.”
His team is now confident that the eCommerce systems can handle even colossal traffic spikes. They can replicate extreme load conditions realistically, scaling to 100,000s of virtual users across multiple geographies and mobile networks — at a fraction of the cost they had been spending with their legacy tool. NeoLoad lets them dynamically right size their testing infrastructure on demand. It’s not uncommon for customers to realize savings of up to 80% of their cloud testing resource costs. NeoLoad also natively interacts with container orchestrators such as Kubernetes, OpenShift, Microsoft AKS, Amazon EKS and Google GKE to provision and use load testing infrastructure automatically, on-demand.
“More frequent, realistic testing means regular tuning on the fly. We no longer just test during holidays, it’s happening throughout the year,” says Ajay.
Bottom line: With NeoLoad, Abercrombie & Fitch has successfully prepared for peak holiday traffic, spikes, and surges every year since 2010.