In 2012, Knight Capital Group lost $440 million in 30 minutes when a “large bug” in their software triggered series of unexpected automatic stock trades. As a result, company business continuity required a nearly $400 million emergency investment and an effective change in management control.[i] This example demonstrates a clear correlation between a QA lapse and a subsequent impact on business strategy. So why do corporate boardrooms so often cast quality assurance as a cost center, rather than a strategic priority? Where’s the disconnect that causes QA’s critical role to go unnoticed?
In part, this reflects the corporate reward and recognition systems. For example, CEOs may receive bonuses and accolades when the stock price rockets, but improvements to quality testing don’t always generate the same fanfare. This reality may manifest when funding requests for new QA personnel or technology get rejected. It may surface when development is asked to provide better solutions, faster, without any corresponding improvements – or even reductions – in the supporting quality infrastructure.
In contrast, QA teams and executives often focus on issues such as bug counts and severity, database and performance errors, and customer experience. While these are appropriate metrics within the development team, without context they may not make the sale with top executives. However, it’s mostly semantics – while QA metrics may sound different than C-Suite metrics, they are inseparable.
To be clear, this doesn’t reflect apathy in the executive suite or on the QA team. The disconnect happens when QA management and business executives don’t speak the same language. This leads to confusion and obscures the deep connection between QA metrics, such as test effectiveness or bug severity trends — and metrics the C-suite cares about, such as stockholder value, market share and profitability.
As a result, it’s unclear to executive decision-makers how a quality initiative will deliver acceptable returns and strategic value. They may recognize the QA initiative solves a problem, but they may not see how it addresses their most important problems. But that perception can change.
By proactively aligning QA results and reporting with larger business strategies, QA management can effectively communicate upwards and secure a better response to its initiatives.
To start, QA must understand strategic business priorities and how they are served or threatened. There’s also a need to identify dependencies these goals have in descending levels of the org chart.
Senior executives focus on protecting, sustaining and growing the business and shareholder value. Each step in this hierarchy encapsulates component objectives. For example, protecting the business may rely on the attainment of risk management, legal compliance and security targets. The next tier of objectives that sustain the business may depend on factors such as operational efficiency, market share and public goodwill. Finally, top-level growth objectives may be driven by improvements in revenue, brand value, stock price and innovation.
Each of these priorities can be further subdivided into tasks for subordinate levels of the organization. Strategic revenue and brand goals rely on specific sales and marketing strategies. In turn, these strategies may depend upon timely and competitive product releases. Protecting the business may demand mitigation of diverse physical, environmental, and cyber threats.
While the objectives may appear to be discrete, they are highly interdependent and reflect similar dynamics. Understanding the links between the QA initiatives and strategic outcomes is a critical first step.
Corporate objectives for market share and revenue don’t occur in a vacuum, they build on the attainment of smaller strategic and tactical objectives. While there’s no guarantee how each QA result will impact business strategy, there is a strong collective relationship. If products are unreliable, late to market, or don’t meet customer needs, they’re unlikely to make a positive contribution to market, revenue or related strategies.
Still, QA is often seen as a business cost rather than a strategic partner. For QA to change the narrative, it must understand and believe in the connection and value delivered. If QA leadership is not sold on the benefits, no one else will be.
So what QA metrics make the best case in the C-suite? QA metrics are just that – QA metrics. Some translation and positioning are needed to show their relevance to strategic goals. QA metrics with the greatest impact on strategic objectives generally relate to QA speed, results, and cost trends. While individual QA events may be interesting, sustained trends will have the greatest significance. Trends also provide the most meaningful comparison of different development methodologies and teams.
Metrics related to QA and development speed are particularly relevant. For example, the average time for ideas to go from concept to delivery directly affects the company’s ability to solve customer problems on time. In turn, this has direct bearing on strategic goals for revenue, market share, and innovation. Examples of speed-related metrics include:
- Lead Time
- Cycle Time
- Team Velocity (Points per sprint/day)
- Defect Resolution Time
- Active Days (vs. Inactive Time on Administrative tasks)
Metrics related to QA and development output quality often predict the subsequent application security and customer experience. They are strategic precursors for operational efficiency, public goodwill, customer satisfaction and legal compliance. Examples include:
- Open/Close Rates
- Burndown Chart
- Defect Category Rates
- Mean Time Between Failure (MTBF)
- Mean Time to Recover/Prepare (MTTR)
- Application Crash Rate (ACR)
Cost of quality metrics demonstrate QA contributions to operational efficiency, expense management and labor optimization strategies. Examples may demonstrate problem identification early in the development cycle, when issue mitigation is comparatively cheap and non-disruptive. Other candidates include:
- Mean Time to Detect (MTTD)
- Defect Detection Percentage (DDP)
- External Failure Remediation Cost
- Internal Failure Remediation Cost
- Inspection Costs
- Error Prevention Cost
Of course, having great metrics and getting a purchase request approved are not the same thing. With that in mind, here are 5 ways to improve executive response to QA initiatives and requests.
1. Communicate QA value early and often
Chinese strategist Sun Tzu said, “Every battle is won or lost before it’s ever fought”. While it never hurts for QA to make its case when requesting purchase approval, it’s much better if QA value has been consistently established in preceding months.
2. Report on things that matter to the audience, in terms they understand
This facilitates better understanding, engagement and response. Convey directional trends for QA speed, results, and cost. Align QA outcomes with the perceived strategic priorities of the relevant organizational decision level.
3. Focus on outcomes more than process
QA process improvements may have intrinsic value, but it’s important to show a plausible connection to customer experience, security, or other strategic outcomes.
4. Keep it real
Quality assurance has a significant impact on strategic priorities, but there is not a perfect cause-and-effect correlation. Focus on trend and pattern data; the impact of individual QA events may be unpredictable, but sustained trends and milestones tell a clearer story.
5. Make good use of analytic data
Meaningful QA analytics and engaging, visual dashboards make the case clear, memorable and compelling.
Smart quality assurance investments enable teams to test smarter, seamlessly, and at scale. They position QA to be a key enabler of business strategy, by optimizing time-to-market, customer experience and risk management. By understanding and communicating QA’s critical role, and the correlation between testing and strategic results, QA leadership can capture the business value of software quality and build executive support for its initiatives.