As a testing professional, you probably strive for perfection. You likely take pride in your work and want your company to have the best products on the market. To achieve that, you and your team must track and monitor an array of metrics to help optimize your operations and set your business up for success.
For example, you may be keeping track of the number of tests that you run, your requirements coverage, what percentage of tests are automated vs. manual and the pass/fail rates of each test run. And, of course, that’s just the tip of the iceberg. There’s no shortage of quality metrics to track, measure and report.
While all of these testing KPIs are important to you and to running your team as efficiently and effectively as possible, that doesn’t necessarily mean that they’re things your C-Suite cares about. Although quality assurance is essential to the success of any product, it’s not uncommon for others outside of the technology organization to ignore it until there’s a problem. Like it or not, if the tremendous value that you provide by doing your job well is taken for granted, chances are that the metrics and key performance indicators (KPIs) that you’re so diligently tracking will be too.
That means that when it comes to communicating with the C-Suite, you’re not going to get very far if you go too deep into the weeds. If you want to get their attention and demonstrate your team’s value, you instead need to focus in on the big-picture metrics that directly relate to business success. In this article, we’ll take a look at what those metrics are, and at some basic steps that you can take to ensure that you’re setting yourself up to get the best results for each.
The testing KPIs your C-Suite cares about
When you talk to your C-Suite about quality assurance, chances are that there will be three main things that will get their attention:
1. Speed to market
First and foremost, executives want to know how quickly they can bring new features to market. After all, the faster new features are delivered, the faster your company can respond to changing customer expectations and market opportunities.
Of course, testing can increase the amount of time between the writing and delivery of new code. Anything that you and your team can do to expedite the process and shorten testing time is going to be seen positively. Demonstrate your success to business leaders by reporting on how long testing takes your team vs. industry average, and ideally how your testing efficiency is increasing over time.
If testing is a bottleneck that’s delaying your company’s ability to deliver new code, that’s something you need to address. Approach test automation strategically, reduce test suite bloat and get involved in the development cycle earlier so you can write test cases earlier and more closely align them with business requirements. If you lack the tools you need to streamline testing, now is the time to bring it up with leadership. Making this request in the context of your goals, and how you could better support the business, will increase your odds of getting the necessary stamp of approval from executives.
2. The number of bugs in production
While you may be intently focused on a goal of zero production defects, the fact is that there will always be bugs. It’s just the reality. And the bigger the product, the more scenarios there are to test, which means the more bugs that could potentially slip through.
But not all bugs are equal in the eyes of your executives. While they too want your products to have as few defects as possible, they’re really only interested in the ones that can negatively affect customer experience. They’re going to want to know what they are, how they were caught before production and what you’re doing to prevent similar ones in the future.
That’s important to keep in mind because when you’re reporting on the number of defects in production, you don’t want to bog the C-Suite down with lower-level issues — think formatting errors, for example — that they won’t consider as critical. Instead, track the relative severity of bugs and focus your attention on the most serious among them that are most likely to lead to attrition. Focusing on the highest-risk areas can both help you increase efficiency and demonstrate your value to executives, by showing that testing is aligned with their goals and concerns.
3. The amount of money being spent on testing
The C-Suite always has its sights on the bottom line, which includes how much money your company is spending on testing. To understand the value of the return they are getting on their investment, executives are going to want to know how many bugs were found in production versus how many tests were executed for every release. Although they understand the importance of testing and can accept that it comes at a cost, they want to make sure that the company is going about it efficiently.
To maximize your efficiency, you may need to experiment with pulling different levers. For example, you might need to scale test automation, or you might need to pare it down to the most effective areas so you can focus more time on exploratory testing. You’ll also want to do some analysis about where defects are most commonly found in production so that you can focus your attention on those areas.
By regularly reporting on these metrics, you’ll be able to keep your executives informed about the aspects of quality assurance that they care about most. That will save everyone time and help reinforce the tremendous value that testing provides.
Act like a tester, communicate like an executive
Testing is an essential part of deploying new software and features, and one that your business cannot succeed without. To be successful at it, you need to track a variety of KPIs. Yet to help executives see the real value of what you’re doing, you need to speak their language. That means talking to them in terms of the metrics that they’re most interested in: speed to market, the number of bugs in production and cost.
If you can report these testing KPIs back to them, ideally within the context of several releases, you’ll be able to get through to the C-Suite and demonstrate the impact that you’re having.