Headquartered in New Albany, Ohio, Abercrombie & Fitch Co. is a Fortune 1000 specialty fashion retailer with over 730 stores worldwide, and three offshoot brands under management, including Hollister Co., abercrombie kids, and Gilly Hicks.
In the retail sector, survival depends on the ability to adapt quickly to customer needs. Most recently, every retailer has had to grapple with the sudden seismic shift in consumer behavior due to the COVID-related social-distancing mandates and stay-at-home orders. Online shopping has skyrocketed, and customers increasingly demand a fast and reliable digital experience every time, from anywhere, on any device. While these new habits accelerated trends already under way, PwC research shows that these changes are here to stay. Excellent digital performance has become a do-or-die competitive differentiator.
Abercrombie has always been at the forefront of reading the market and reacting to customers’ evolving needs. Its digital footprint includes 40+ eCommerce desktop and mobile sites, a volume that calls for constant focus and effort to maintain performance at every turn. But it was well prepared to meet the digital transformation onslaught — digital helped the company realize its best income performance in over a decade. Digital sales grew by 39% in 2020, to $1.7 billion — more than half of total sales.
“The reality is, true holiday readiness was not easily achievable before NeoLoad.”
– Ajay Pedhagandham, Quality Assurance Manager, Abercrombie & Fitch
Preparing for online holiday shopping peaks
Probably nothing challenges eCommerce performance engineers as much as preparing for the holiday shopping season. The stakes are high, and systems are ever more complex. Consider:
- As much as 20–30% of a retailer’s annual revenue can come from the holiday season
- Over half of visitors abandon a website that takes longer than three seconds to load — and 80% of them will never come back
- 57% of customers stop buying from a company permanently because a competitor provides a better experience
- Each customer transaction hits at least 82 different technologies — any one of which could be a single point of failure, create a bottleneck, or degrade customer experience
Ajay Pedhagandham, Quality Assurance Manager at Abercrombie & Fitch, manages the performance testing team and is responsible for performance test automation. Most important, he’s been at the helm leading the company’s holiday readiness testing team for the past eight years. He notes that any page-response delay, even for a second, can mean the difference between growth and loss. To prepare for the peak traffic loads, his team has to simulate tens of thousands of orders per hour to ensure readiness. He adds, “Our customers are socially active influencers — a bad shopping experience can lead to a firestorm of negative conversation, which can quickly tarnish the brand.”
- Expand test coverage to meet business growth
- Test faster to respond to rapidly changing customer behavior
- Create realistic “extreme” load tests for peak holiday volume
- Simplify test script maintenance
- Obtain more accurate test results
- Test both front and backend systems at scale
- Reduce infrastructure costs of high-volume load tests
Given the value that Abercrombie places on performance, it’s no surprise that the challenges Ajay encountered with a legacy performance testing tool revolve around cost and coverage. He explains, “As we were growing and expanding our application delivery practices, which included increased global reach, we knew our performance testing coverage had to support this growth. Our performance testing had to be conducted realistically, so that we could prepare for peak holiday volume. The more our virtual user count needs grew, the more we had to pay. It was becoming cost prohibitive to test under real peak scenarios. Plus, time and again, when we were trying to replicate peak load, we would get a lot of false positives. The reality is, true holiday readiness was not easily achievable.”
Abercrombie & Fitch’s search for a new performance testing solution began in 2009 on the heels of its annual holiday readiness evaluation, which had identified gaps with its legacy solution. The wish list was simple — the company wanted a solution that was both cost-effective and flexible/scalable enough to facilitate realistic performance/load testing out of the box. In addition, the team wanted script maintenance simplicity, something it did not have with its existing performance testing tool.
- Met peak order volume testing “success threshold” every year since 2010
- Test design/maintenance reduced by 80%
- Simulates 10,000s orders/hr. across 40+ eCommerce sites
- Expanded performance test coverage to backend systems
- Reduced performance testing costs
Abercrombie & Fitch determined that Tricentis NeoLoad offered the best combination of speed and accuracy. Scripting time/test maintenance has been reduced by over 80% — from seven days down to one. Ajay’s team has eliminated the time-consuming complexity of hand-coding tests, and automated script maintenance means they no longer have to rewrite scripts from scratch every time code changes.
“We’re now walking into the war room having achieved our performance testing goals already, and this enables us to focus on backend systems,” says Ajay. This is crucial because while frontend traffic is upwards of 3X more than usual on peak shopping days, the backend load increases by 10X or more. “This extra time allowed us to identify and fix an issue we never would have caught, or had the time to address, had we been focused exclusively on holiday load testing goals.”
His team is now confident that the eCommerce systems can handle even colossal traffic spikes. They can replicate extreme load conditions realistically, scaling to 100,000s of virtual users across multiple geographies and mobile networks — at a fraction of the cost they had been spending with their legacy tool. NeoLoad lets them dynamically right size their testing infrastructure on demand. It’s not uncommon for customers to realize savings of up to 80% of their cloud testing resource costs. NeoLoad also natively interacts with container orchestrators such as Kubernetes, OpenShift, Microsoft AKS, Amazon EKS and Google GKE to provision and use load testing infrastructure automatically, on-demand.
“More frequent, realistic testing means regular tuning on the fly. We no longer just test during holidays, it’s happening throughout the year,” says Ajay.
Bottom line: With NeoLoad, Abercrombie & Fitch has successfully prepared for peak holiday traffic, spikes, and surges every year since 2010.
- Tricentis NeoLoad
“Peak order volume testing with NeoLoad has met the ‘success’ threshold each year since 2010.”
– Ajay Pedhagandham, Quality Assurance Manager, Abercrombie & Fitch