Why performance validation is an infrastructure issue, too

At Datadog DASH in the spring, the Tricentis NeoLoad team met folks in all types of roles – developers, test engineers, CoE leads, and SREs – and one recurring theme we found was that SREs often didn’t know much about how performance validation happens at their companies.

That’s a fair division of labor. Seemingly, the performance team’s work would be related but not mission-critical to the infrastructure team’s. But that’s not entirely true.

Performance validation is designed to resolve two major infrastructure issues: system outages and cloud overspending. These issues torment the business in different ways, and agentic AI is taking them to a new level.

The latest from NeoLoad: How NeoLoad 2026.2 is empowering scaled performance workflows

Load testing alone wasn’t enough before AI, but it’s definitely not enough now

Picture a major event: Black Friday, Super Bowl Sunday after a company ad airs, or a major product launch. All eyes are on the website or application, and expectations are high. It would be a wildly inopportune time for an outage, but if the performance testing process before go-live isn’t meticulous, that’s exactly what could happen. And when it happens, the five-alarm fire is in the infrastructure team’s house.

Let’s say you’re able to hypothesize that, on the week of the big launch, you’ll need enough compute to support 5x the normal number of requests, so you build that in, load test it, and call it good.

If only it were that easy.

In these situations, teams frequently load and stress test the system, which is a critical step, but it’s not a replacement for a whole strategy. These types of testing don’t tell you whether the system can sustain additional capacity for a sustained period (soak testing), whether something new has degraded since the last check (regression testing), or how the application recovers from a breakdown (resilience testing).

The risk of passing a load test and still being caught off guard is compounding. AI-generated code helps teams build software faster, but it also has 1.7 times more issues than human-written code, according to a CodeRabbit analysis. That higher error rate combined with more code written faster has a snowball effect. Teams aren’t producing a ton of code and then setting it aside until performance engineers can get to it.

Capacity planning for non-human behavior

At large enterprise organizations, performance teams have their finger on the pulse of usage and capacity. Like a grocery store manager, they make sure the store is staffed up for peak-traffic hours and staffed down when business is slow.

But unlike humans, AI agents and automated model training pipelines don’t sleep. Traditional infrastructure scaling models rely on predictable human cycles – business hours, sleeping hours, holidays – but AI breaks that model in two ways.

First, AI agents run continuously: model training jobs and automated pipelines don’t follow human schedules. A training job might kick off at 3 a.m. and consume more compute than your entire business hours load. The grocery store has never in its history had a major rush at 3 a.m., and now there’s a line of agents out the door.

More: 3 NeoLoad MCP use cases

Second, AI is generating unpredictable usage spikes. If a new AI feature ships and nobody knows how compute-intensive it is at scale, the infrastructure team will find out when the bill arrives. The failure to match infrastructure capacity with real-world usage is contributing to the waste of 29% of large enterprise cloud compute budgets, according to Flexera’s 2026 State of the Cloud Report.

Guesswork has become an unaffordable methodology for predicting capacity requirements.

Next steps: Get to know your friendly neighborhood performance engineer

It seems that software is always finding new and interesting types of fire drills to inflict on infrastructure teams, especially today.

The good news is that performance engineers exist precisely to answer the questions that keep infrastructure teams up at night: How much load can we handle? What changed since last time, and does it matter?

But they can’t answer those questions in isolation. The best performance test plans are built in partnership with infrastructure teams, who can provide a clear picture of traffic history, scaling thresholds, known failure points, and the AI workloads that haven’t been modeled yet.

The relationship works when both sides show up to it. Infrastructure teams bring the operational picture, and performance engineers use it to build a test strategy. As a result, infrastructure teams have a clear view of capacity limits, degradation patterns, and the specific conditions under which things break.

For most organizations, that conversation just isn’t happening yet.

Webinar: From shift left to shift right: How NeoLoad enables a continuous performance engineering loop