continuous testing what next


Five Software Testing Best Practices for the DevOps Age

To succeed in today’s digital business environment, application leaders must build teams, toolsets and processes to support shorter release timelines without leaking defects into production. Delivering high-quality software at an accelerated pace requires quality practices to be continuous, highly automated, and to occur throughout the software development lifecycle.

So how can your team get there, and where should you begin? Here are five best practices for optimizing software testing in DevOps environments.

1. Address the Disconnect Between QA and Developers

QA is critical to the software release process. So why is it often thought about last when an organization embarks on a DevOps transformation initiative? There is a common misconception about QA’s role in a DevOps environment: that with continuous integration and continuous delivery, all quality efforts can be automated too.

While it’s true organizations are achieving a higher level of test automation than ever before, release readiness is a business decision – not a technical decision. And while automation can give you a high level of confidence that the release is ready for customers, good test results shouldn’t be confused with release readiness. So how can you address the disconnect and ensure quality is embedded in DevOps processes? Include testers as part of your integrated delivery team.

2. Iteratively Adopt Practices and Tools for Each DevOps Stage

Rather than a rip-and-replace approach to tooling, it’s possible – and even preferred – to iteratively adopt practices and tools for each DevOps stage. To begin, leaders should address the biggest problem areas first. This is a much more approachable method, as it allows teams to adopt tools and practices over time, as their processes mature and needs evolve.

According to recent research from Gartner,** as an application leader, you should Drive your DevOps teams to develop their practices and toolkit iteratively by identifying the biggest constraint in their processes and selecting the quality tools that best help them optimize for that constraint.

The analysts explain: Investing in new technologies requires upfront capital expense, maintenance costs, and the time to train, onboard and operationalize the tools. Application leaders engaging in procuring quality tools for their DevOps toolchains should align business objectives to the selection criteria to ensure the investment leads to business outcomes.

3. Adopt Continuous Testing Practices

A recent study found that organizations with successful agile and DevOps initiatives do a number of things differently than their peers – including active efforts to transform their software testing practices into continuous testing practices. The study, which was led by Forrester analyst Diego Lo Giudice and commissioned by Tricentis, found that these organizations are adopting five core continuous testing practices: allocating proper testing budgets and focusing on improving skills; implementing continuous testing to meet the demands of release frequency; including automating end-to-end functional testing; integrating testers into cross-functional teams and practicing shift-left testing.

Haven’t adopted all five (or any) continuous testing practices successfully just yet? You’re not alone. According to the report, only 26 percent of firms with active DevOps adoptions are currently practicing all five continuous testing practices.

4. Automate Testing Where it Makes Sense

It can be difficult to determine the ideal level of test automation your team should be striving for during the DevOps transition. But with releases happening significantly more frequently, the volume of testing must also increase significantly. That’s where automated testing comes into play. According to recent research, successful DevOps shops are significantly more likely to have high levels of automation for key testing and QA processes (test case design, functional test automation, test data management, etc.).

However, teams shouldn’t simply proceed with the goal of automating as much as possible. Instead, the goal should be to lean on the team’s more tenured testers to develop a test automation strategy that maximizes resources and eliminates the need for testers to manually perform repetitive test runs. By eliminating rote tasks, automation allows testers to spend their time more strategically – both planning and executing automation strategies and performing the exploratory testing that cannot be replaced by automation

5. Focus on the Right Metrics

Determining the test cases that make the most sense to automate comes down to having good test data. Centralizing QA metrics to get a holistic view of all of your tests-across all of the delivery pipelines- will help you see how frequently tests are failing, how long they take to run, and how much time you are spending on environment setup and data creation.

Once you’ve achieved a relatively high level of automation, the ability to quickly view and analyze metrics will become much more important. Access to the right metrics will allow you to gain insights that help you focus test automation strategies on the areas in which it can have the most significant impact for your business. So what are the right testing metrics? According to Forrester, agile and DevOps experts consistently rank risk-related metrics among their top 3 most valuable metrics. Organizations with mature agile and DevOps practices also tend to focus on contextual metrics such as requirements coverage, rather than simply counting the number of tests run or defects found.

**Gartner, Build Continuous Quality into Your DevOps Toolchain, Jim Scheibmeir, Thomas E. Murphy, Joachim Herschmann, Christopher Little, March 13, 2018