Why 95% of AI pilots fail – and what it takes to scale in the agentic era

Last August, MIT released a landmark report that confirmed what many enterprise leaders had started to fear: most AI pilots are failing. After reviewing hundreds of AI initiatives, researchers found that 95% of generative AI pilots failed to reach production or deliver measurable results. The headline quickly hardened into a cliché: AI doesn’t scale.

But there was a bright spot buried in the report – the 5% of pilots that succeeded. The successful implementations shared a pattern. They were integrated into core workflows, capable of learning, and adapted to feedback. The report suggests that agentic AI, built with memory and specialized for workflow execution, naturally reflects these traits and could help close the gap between failed generative AI experiments and enterprise-wide adoption.

Yet agentic AI can still fall prey to the trappings of the report’s 95% – generic training, siloed implementation, inflated expectations. This post explores MIT’s findings about what “good” AI implementations look like and highlights examples using Tricentis AI tools to show how the right approach can help QA teams scale beyond AI experiments and finally see real returns.

Agentic AI can’t scale in silos

An individual AI agent is powerful, but inherently restricted. By design, an agent executes only a specific set of tasks with precision. On its own, it can’t achieve what the most successful pilots in the MIT study did: integrate with workflows at scale.

Successful implementations in the report treated AI like infrastructure instead of a single tool, deeply embedding it in core business processes. The same should be true for any agentic AI project. An agent’s impact can be transformative — and scalable — when it can coordinate and share contexts with other agents across multiple workflows.

Earlier this month, Tricentis announced its new Agentic Quality Engineering Platform, powered by an AI Workspace where QA teams can build, govern, and orchestrate custom and Tricentis-built agents across quality domains. The platform breaks down siloed AI, managing and coordinating agents that span quality processes, from test creation, management, and analysis.

By orchestrating agents that are directly embedded in quality workflows, AI Workspace gives teams the foundation they need to scale AI across business processes – the same pattern that distinguished successful implementations in the MIT report.

Context as a competitive advantage

Some AI pilots in the MIT study did see employee uptake – but for the wrong tasks, and with the wrong tools. Personal AI tools like ChatGPT “won the war for simple work,” helping employees fast-track manual tasks but failing to produce meaningful value for enterprise-wide projects.

Generative AI tools struggle to scale because most can’t learn. Users have to restate context with every prompt, which is manageable for drafting an email but a serious limitation in multi-step processes.

Agentic AI closes this gap. Agents store context, learn from interactions, and adapt over time. However, even with stronger memory, agents must first be trained on and store the right, domain-specific data to be effective. Otherwise, like generative AI, they become too generic to provide much value. Agents are only as good as the data – and the platform – they are built on.

Tricentis agents are purpose-built for quality engineering, grounded in Tricentis’ 20 years of QA leadership. Take one Tricentis agentic solution, Agentic Test Automation, as an example. It is built on Tosca, a leading test automation tool for more than a decade, and uses Tosca’s proven technology and automation engine to autonomously create complete and stable test cases. With Tosca’s deep expertise as its foundation, the solution is set up to execute quality tasks with precision.

Tricentis AI Workspace is also purpose-built for QA. Rather than serving as a generic orchestration layer, it uses Tricentis’ experience to embed QE best practices into all its agent workflows and governs them within a single, specialized environment. This is why the MIT report found that the companies partnering with vendors for their AI pilots were twice as likely to succeed than internal builds — providers like Tricentis are better positioned to create specialized AI that is already attuned to industry-specific workflows and policies.

Agentic AI and the humans-in-the-loop

Learning and adapting over time sets agentic AI apart from the underwhelming generative AI tools highlighted in the MIT report. Agents can improve continuously, with human interaction playing a critical role in the learning process. User guidance refines and controls AI performance, keeping people central to adoption even as agents grow more autonomous.

In some of the failed pilots, AI was treated by employees as a bolt-on efficiency layer: a generic tool for isolated tasks that didn’t merit a larger governance, orchestration, or accountability structure. This kept these tools from enterprise-wide adoption. Transformative AI initiatives require a clear framework, one that directs how AI should operate, evolve, and align with a company’s workflows and policies.

AI Workspace brings together agentic coordination and human oversight. Teams can define standards and policies, ensuring agents operate independently within established boundaries and escalate to humans when judgement is needed. By coordinating agents to execute tasks, it enables humans to focus on crafting the strategy and direction that eventually guides them.

Without such a framework, AI can be stuck as just an add-on, one that doesn’t result in real return. Centralized, human-led oversight and governance help push AI past the pilot, giving it the guardrails it needs to scale.

Making AI work for enterprise quality organizations

MIT’s findings outline the next steps for enterprises looking for better AI: stop relying on prompt-heavy tools, partner with strategic vendors, and prioritize workflow integration. Researchers point to agentic AI as a new frontier for AI, a technology that could help companies out of pilot purgatory.

Tricentis is building this future for quality assurance, providing the industry with an integrated agentic platform powered by specialized agents. The solution redefines AI for quality, giving teams the structure to scale AI from a side project to an operational advantage. For QA teams ready to see real returns, the agentic quality engineering platform is where AI finally pays off. To learn more about the new platform and our team of agents, check out our recorded launch event, Powering what’s next in AI-driven quality engineering.