New Critical Defects [Functional Validation]

Software Testing 101

New Critical Defects [Functional Validation]

What is “New Critical Defects”?

The  “New Critical Defects” DevOps quality metric goes beyond simply counting the number of automated tests and introduces the concept of risk. Teams using this metric have business risks clearly defined and tests correlated to those risks. With this insight, they can focus their test automation efforts on tests that matter most to the business. 

How “New Critical Defects” Ranks vs. Other Quality Metrics

In new Forrester research on DevOps quality metrics, DevOps leaders ranked “New Critical Defects” as the #12 most valued functional validation quality metric. Of the 24% of DevOps leaders who measured this metric, 47% of them ranked it as one of their most valuable DevOps quality metrics. This metric was not found to be a “DevOps Differentiator” (DevOps experts/leaders measured it significantly more than DevOps laggards did).

Here‘s a quick look at how “New Critical Defects” is positioned vs. other Functional Validation Metrics—based on the raw data collected from DevOps experts. “DevOps Differentiators” are highlighted in green.

These findings are expanded upon in the new ebook: Forrester Research on DevOps Quality Metrics that Matter: 75 Common Metrics—Ranked by Industry Experts.