What is DevOps Testing?

Software testing 101

What is DevOps Testing?

With DevOps, many applications are now released anywhere from every two weeks to thousands of time a day. As a result, the time available for test design, maintenance, and especially execution decreases dramatically.

If your organization has adopted DevOps and is performing Continuous Delivery, software may be released hourly—or even more frequently. In this case, feedback at each stage of the process can’t just be “fast”; it must be nearly instantaneous. If quality is not a top concern for your application (e.g., if there are minimal repercussions to doing a rollback when defects are discovered in production), running some quick unit tests and smoke tests on each release might suffice. However, if the business wants to minimize the risk of faulty software reaching an end user, you need some way to achieve the necessary level of risk coverage and testing breadth—fast.

For DevOps test automation, there are several significant impacts:

  • Testing must become integral to the development process (rather than a “hygiene task” tacked on when development is complete)
  • Tests must be ready to run almost as soon as the related functionality is implemented
  • The organization must have a way to determine the right tests to execute at different stages of the delivery pipeline (smoke testing upon check-in, API/message layer testing after integration, and end-to-end testing at the system level)
  • Each set of tests must execute fast enough that it does not create a bottleneck at the associated stage of the software delivery pipeline
  • A way to stabilize the test environment is needed to prevent frequent changes from causing an overwhelming number of false positives

Here are some ways for testers need to address time pressures:

  • Identify which test cases are critical for addressing top business risks
  • Define and evolve tests as the application constantly changes
  • Rebalance the test pyramid so that most tests execute at the API layer, which is at least 100 times faster than UI test execution
  • Integrate tests into the delivery pipeline
  • Run distributed tests across multiple VMs, network computers, or in the cloud as appropriate
  • Enlist service virtualization and synthetic data generation/TDM so that testing doesn’t need to wait on data or environment provisioning

Preparing for DevOps Testing

Many organizations today are adopting DevOps to accelerate software delivery. However, once they have invested significant time and money optimizing most parts of the software delivery process, testing holds them back from achieving the desired results. Why? Because DevOps test automation is still dominated by yesterday’s tools and processes—which don’t meet the needs of today’s accelerated development processes.

How can you ensure that you and your team help the organization achieve its objectives? Continuous Testing will be key. By Continuous Testing, we don’t just mean “Test Automation.” We mean the process of executing automated tests as part of the software delivery pipeline in order to obtain feedback on the business risks associated with a software release candidate as rapidly as possible.

Watch the Preparing for DevOps Testing  on-demand webinar to explore why Continuous Testing is essential for DevOps. You’ll learn:

  • What Continuous Testing requires from the DevOps tester
  • The steps you can take to ensure that you’re prepared for this transformation
  • How the role of QA can and should shift from the reactive role of validating requirements to a more proactive “quality engineering” role that involves protecting and optimizing the business

Reality Check: The Role of Manual Testing in DevOps Testing

DevOps practices have helped enterprise organizations meet demands to deliver new software faster. But if this newfound speed is at the expense of software quality, the result can put the business at considerable risk. Testing practices and DevOps testing tools have been evolving to keep up with this new pace, often starting with the reduction of manual testing.

Does that mean that manual testing has no place at all in testing practices of the future? Would enterprise organizations eliminate manual testing entirely if they could? Watch the Reality Check- Role of Manual Testing in DevOps on-demand  webinar to see why the adoption of DevOps won’t mean the death of manual testing.

You will learn:

  • The changing role of manual testing and UI test automation
  • What teams can do to meet required levels of testing, without compromising on speed
  • What these changes mean for testing service providers

Tricentis Tosca

Exploring software testing tools? Try the industry leader for GUI/API testing, exploratory testing, test data management, risk-based testing, and more. Read more.

From Manual Testing To Continuous Testing [Webinar Recap]

Expectations for Continuous Testing are making test automation even more vital—and, in many senses, more challenging. How can manual testers meet these new demands? Read more.

What Is Continuous Testing

What is Continuous Testing, what does it mean for Continuous Testing in DevOps, Agile, Continuous Integration & Continuous Delivery, and how is it different than QA test automation? Read more.