What is “Unit Test Pass/Fail Rate”?
The “Unit Test Pass/Fail Rate” DevOps quality metric measures the percentage of unit tests that pass or fail. It is notoriously easy to manipulate. For example, the team can increase the pass rate (or reduce the fail rate) by simply adding redundant passing tests that do not add value.
How “Unit Test Pass/Fail Rate” Ranks vs. Other DevOps Quality Metrics
In new Forrester research on DevOps quality metrics, DevOps leaders ranked “Unit Test Pass/Fail Rate” as the #3 most valued build quality metric. Of the 30% of DevOps leaders who measured this metric, 60% of them ranked it as one of their most valuable DevOps quality metrics. This metric was not found to be a “DevOps Differentiator” (DevOps experts/leaders measured it significantly more than DevOps laggards did).
Here‘s a quick look at how “Unit Test Pass/Fail Rate” is positioned vs. other Build Metrics—based on the raw data collected from DevOps experts. “DevOps Differentiators” are highlighted in green.
These findings are expanded upon in the new ebook: Forrester Research on DevOps Quality Metrics that Matter: 75 Common Metrics—Ranked by Industry Experts.