What is “Number of Automated Tests”?
The “Number of Automated Tests” DevOps quality metric counts the number of automated tests performed for the specified build. However, it does not account for test redundancy, test effectiveness, or test flakiness. Having more tests does not necessarily add more value, nor does it automatically increase a team’s chances of releasing faster.
How “Number of Automated Tests” Ranks vs. Other Quality Metrics
In new Forrester research on DevOps quality metrics, DevOps leaders ranked “Number of Automated Tests” as the #17 most valued build quality metric. Of the 34% of DevOps leaders who measured this metric, 44% of them ranked it as one of their most valuable DevOps quality metrics. This metric was found to be a “DevOps Differentiator” (DevOps experts/leaders measured it significantly more than DevOps laggards did).
Here‘s a quick look at how “Number of Automated Tests” is positioned vs. other Build Metrics—based on the raw data collected from DevOps experts. “DevOps Differentiators” are highlighted in green.
These findings are expanded upon in the new ebook: Forrester Research on DevOps Quality Metrics that Matter: 75 Common Metrics—Ranked by Industry Experts.