“One of the key changes is that we got the business involved [in performance testing]. We’d hear that the performance was ‘rubbish,’ but what does that mean? So we got them to define the non-functional requirements in terms of how long something should take, and we got them to give us some critical workflow scenarios. Then we ran tests and measured results against those SLAs,” says Botley.
Adds Sewell, “This is the first place I’ve worked where we do performance testing properly. And NeoLoad allows us to do that for every step, all non-functional requirements.”
With its modern, iterative approach to performance testing, the FA can now handle 10X more concurrent users accessing its website. Previously, a queuing tool was needed to throttle the volume of traffic, as there was no confidence that the system could handle more than 400 people at a time. “Now we’ve tested up to 4,000 users,” says Sewell. “We’re confident in that number. NeoLoad gives us the benefit of not worrying about that. When people visit our site, there is not a performance issue in them doing what they are trying to do.”
The FA takes the same iterative approach to infrastructure optimization. The association is migrating its on-prem systems to the cloud (Microsoft Azure). “We went through a whole process of experimentation on the different settings — CPU, memory, whatever — that would best suit our needs. And NeoLoad would test against those different settings. We’d keep turning or adjusting till it reached the point of being efficient and working in the way that we needed it to,” says Sewell. After all, scalability isn’t free.