Almost exactly one year ago, Forrester confidently predicted that 2018 would be “the year of Enterprise DevOps.” The blog, authored by the late Robert Stroud, began:
DevOps has reached “Escape Velocity.” The questions and discussions with clients have shifted from “What is DevOps?” to “How do I implement at scale?”
Continuous Testing is not far behind. In early 2014, SD Times proclaimed “Forget ‘Continuous Integration’—the buzzword is now ‘Continuous Testing’” (in the very first article in the publication’s Continuous Testing category). At the time, the concept of Continuous Testing seemed about as far-fetched as a Silicon Valley snowstorm to most testers in enterprise organizations—where pockets of DevOps were just surfacing among teams working on “systems of engagement.”
But since 2014, the world has changed. As Forrester predicted, the vast majority of enterprise organizations are now actively practicing and scaling DevOps. And the larger focus on Digital Disruption means that it’s now impacting all IT-related operations: including systems of record as well as systems of engagement.
When ExxonMobil QA Manager Ann Lewis so memorably asked “Is it all just a bunch of hype? Really?” at the Accelerate 2018 Continuous Testing conference, the clear consensus was a resounding “no.” Digital Transformation, DevOps and Continuous Testing have gotten real for the conference attendees, largely composed of QA leaders across Global 2000 organizations. So real, in fact, that their employers cleared their schedules for a week and sent them to Vienna to learn what’s really needed to achieve Continuous Testing for DevOps…in an enterprise environment.
Here are some of the key lessons learned—shared by leading testing professionals that have already made Continuous Testing for DevOps a reality in their own organizations:
“Test Data is a Pain the Ass”
Renee Tillet, Manager of DevOps Verify at Duke Energy, offered her perspective on one of the most underestimated pains of Continuous Testing: Test Data Management. Renee asserted:
“If you’re doing test automation, what’s the biggest pain in your ass? It’s test data. We would be in the middle of our sprint—the developers are done, the testers are getting ready to test, and guess what? The tester has no test data. Not only does he not have test data, but he doesn’t have time to go create that test data now. It’s too late.
By the time you get to that user story, your definition of ready should include not just what the developer needs, but also the test data you need to verify it. The test plan needs to be ready, and the data needs to be in the environment—or we don’t accept that story into the sprint.
Initially, we would create parameterized test cases, we’d put data in them, and they would run in the Dev environment. But then we’d try to run them over in the test environment, which was the next higher environment, and they would fail because the data was different. So, we came up with a data strategy that allowed us to use the same test data in all the environments.”
Number of Test Cases: Less is More
Numerous experts shared that a high number of test cases is no longer something to be worn as a badge of honor. It doesn’t help provide the fast feedback that the team expects…
Continue reading on SD Times: Continuous Testing for DevOps: Is it All Just a Bunch of Hype?
For more insights from industry leaders, browse our video library of selected Accelerate sessions.