Transforming Testing for Agile and DevOps
Enterprise Continuous Testing
Even with the most extreme automation, the “test everything” approach is not feasible—or necessary. If you rethink your approach to testing, you can get a thorough assessment of a release candidate’s business risk with much less testing than you’re probably performing today.
“Enterprise Continuous Testing: Transforming Testing for Agile and DevOps” introduces a Continuous Testing strategy that helps enterprises accelerate and prioritize testing to meet the needs of fast-paced Agile and DevOps initiatives. Software testing has traditionally been the enemy of speed and innovation—a slow, costly process that delays releases while delivering questionable business value. This new strategy helps you test smarter, so testing provides rapid insight into what matters most to the business.
This book is written for senior quality managers and business executives who need to achieve the optimal balance between speed and quality when delivering the software that drives the modern business. It provides a roadmap for how to accelerate delivery with high confidence and low business risk. In summary: If you want to realign your Global 2000 organization’s quality process with the unrelenting drive towards accelerated delivery speed and “Continuous Everything,” then you’re in the right place.
All proceeds from Enterprise Continuous Testing will go to Specialisterne, a non-profit committed to inclusion through neurodiversity. They partner with corporations, universities, schools, and community agencies to help companies to employ work-ready neurodiverse talent.
Let’s face it. Businesses don’t want—or need—perfect software. They want to deliver innovations as soon as possible. A single delay might be the only opportunity a competitor needs to gain market share and become the new disruptor.
Testing is essential for accelerating the delivery of innovative applications without incurring unacceptable business risk. We need fast feedback on whether the latest innovations will work as expected or crash and burn in production. We also need to know if these changes somehow broke the core functionality that the customer base—and thus the business—depends upon.
However, even with the most extreme automation, we simply don’t have time for the “test everything” approach…If we rethink our testing approach, we can get a thorough assessment of a release candidate’s business risk with much less testing than most companies are doing today…
Today, “Digital Transformation” is driving enterprises to innovate at lightning speed. We need to dedicate resources to creating new sources of customer value while continuously increasing operational agility. Otherwise, we risk waking up one day to find out that even though we did nothing “wrong,” we somehow lost.
The speed of Digital Transformation is already staggering, and it’s only going to increase. To put this into some very concrete terms, consider that:
– There are 7.7 billion people in the world
– 4.5 billion have regular access to a toilet
– 5.5 billion own a mobile phone
All of a sudden, a huge number of people jumped from a very provincial lifestyle straight into digital times—creating a tremendous demand for more, and more innovative, software…
If you’ve ever looked at test results, you’ve probably seen something like this:
What does this really tell you?
You know that…
– There’s a total of 53,274 tests cases
– Almost 80% of those tests (42,278) passed
– Over 19% of them failed
– About 1% did not execute
But…would you be willing to make (or recommend) a release decision based on these results? Maybe the test failures are related to some trivial functionality. Maybe they stem from the most critical functionality: the “engine” of your system. Or, maybe your most critical functionality was not even tested at all…
Many organizations think that this risk-based approach to testing sounds great in theory, but doubt that they can really achieve it—especially given the limited downtime available with today’s unrelenting, fast-paced development iterations. Understanding and targeting the top business risks is actually a relatively simple and painless process. In fact, we’ve found that the less time you spend on it, the better the results.
In this chapter, I’ll explain how you can systematically identify your top risks in a matter of hours. This is a foundational step for 1) determining where to focus your testing efforts and 2) monitoring how well your business risks are tested—and whether they are actually working as expected.
Before diving into how to assess your top risks and cover them as efficiently as possible, let’s first take a step back and consider what risk really means…
As the last chapter outlined, a risk-weighted assessment of your requirements and subrequirements helps you decide where to focus your testing efforts. Based on an assessment like the following, you might decide to start off by testing the subrequirements that have the greatest contribution to the overall risk coverage, then address the other subrequirements as time permits.
Once you know what to test, you need to determine how to test it. Now, you want to focus on achieving the greatest risk coverage for the targeted requirements as efficiently as possible…
To derive the greatest value from a carefully-crafted test design, you must ensure that the planned tests are executed as rapidly and as often as needed. In yesterday’s waterfall development iterations, manual testing was often a viable—though costly—solution. The benefit of automation has always been evident. Yet, with labor arbitrage, the low cost of manual testing allowed it to remain prevalent for much longer than it should. With cost-effective manual testing options at their fingertips, organizations deferred initiatives to build and scale test automation.
Even 5 years ago, only 30% of enterprise software testing was performed fully “in house,” and the vast majority of that testing was not automated. Today, 97% of organizations are practicing Agile to some degree and 73% have active or planned DevOps initiatives. With this fundamental shift, test automation reaches a tipping point. Testers are expected to be embedded in the team and testing is expected to be completed “in-sprint.” With this fundamental shift, test automation reaches a tipping point.
Why does the shift to Agile and DevOps make test automation imperative?…
“Yet again, Wolfgang Platz provides measured and insightful commentary on the ever-changing software testing landscape. Wolfgang’s endless passion and enthusiasm matched with obvious backing of significant research makes this compelling and enjoyable reading. Software testing is in safe hands and takes another leap forward.”
Grant Borrie, Technical Director – TTC Global
“Wolfgang knows what’s required to make Continuous Testing a reality in complex environments. I’ve led numerous enterprise testing transformations based on these core strategies, and can personally attest to their effectiveness. This is a ‘must-read’ for all IT leaders and quality professionals.”
Raj Kanuparthi, CEO and Founder – Narwal
“Wolfgang compiles years of experience in automated testing in a compact and digestible form. If you need to quickly come up to speed or expand an area of your testing knowledge, this is a great book.”
Mitch Ashley, CEO and Founder – Accelerated Strategies Group
“Enterprise Continuous Testing is a short book, but it is packed with valuable information! I highly recommend this book for anyone who wants to create an effective test automation strategy for their team.”
Kristin Jackvony, Blogger – Think Like a Tester
About the Author
Wolfgang Platz is the Founder and Chief Product Officer of Tricentis. Wolfgang is the force behind innovations such as model-based automation and the linear expansion test design methodology. The technology he developed drives Tricentis’ Continuous Testing Platform, which is recognized as the industry’s #1 solution by all top analysts. Today, he is responsible for advancing Tricentis’ vision to make resilient enterprise automation a reality across Global 2000 organizations.
Prior to Tricentis, Wolfgang was at Capgemini as a group head of IT development for one of the world’s largest IT insurance-development projects. There, he was responsible for architecture and implementation of life insurance policies and project management for several projects in banks.
Wolfgang holds a Master’s degree in Technical Physics as well as a Master’s degree in Business Administration from the Vienna University of Technology.