quality engineering header

Agile testing

Continuous Testing Live: What’s Driving the Quality Engineering Evolution?

“Quality engineers, as we talk about them, this new individual or this new entity, are going to have the skills to be able to do development and do testing at the same time.”

– Jeff Wilkinson, Accenture Managing Director

Accenture Managing Director Jeff Wilkinson explains why we hear so much about eliminating manual testing…or even suggestions of getting rid of the term “testing” altogether. Learn what’s changed in modern-day application development, why a new approach to quality engineering is required, and how today’s testers can deliver the biggest impact on the lines of business they support.

Don’t miss a single episode of Continuous Testing Live—subscribe today at iTunes, Google Play, or SoundCloud!

Wayne: Hello, this is Wayne Ariola from Tricentis. I’m here with Jeff Wilkinson, the managing director of North America – Testing Platform Lead. Did I get that right?

Jeff: That’s pretty close.

Wayne: Oh my gosh.

Jeff: It’s very good. Well done.

Wayne: I was really worried about that.

Jeff: I’ll answer to anything, really. I’m not picky.

Wayne: I was really worried about blowing that line. Anyways. Hey, we’re here, and we have an opportunity to talk, and I think the question I have for you, Jeff, is really more about … We are in an immense amount of change. So I’ve been in the testing space for about 20 years, but I feel like there’s an inflection point. It feels like it’s in turmoil. Any insight? What’s going on?

Jeff: I often start my discussions with clients, with workshops, with conferences, by being disruptive and saying, “We are in the business of eliminating manual testing.”

Wayne: Interesting.

Jeff: We’re in the business of eliminating scripted testing. And, in some cases, if I’m really feeling kind of strange, I’ll say, “We would like to eliminate the word testing from the English language.”

Wayne: Really?

Jeff: Maybe I’m just doing that so that I could claim insanity defense if I get arrested later on, but the reality is that testing is not our father’s testing. Today, we’re talking about “quality engineering” and the reason for that is that the way that systems are being developed nowadays is very different than they were a decade ago, even five years ago, right? Because digital has come into play, and every company’s a digital company, it’s forcing a different type of application development.

Wayne: Interesting.

Jeff: It’s forcing continuous delivery, right? If you are a digital company, you want to be on the web. You want a presence out there. You want to get things into production as quickly as you can. You can’t have a slow waterfall approach to doing systems development. Agile development is requiring continuous delivery. Continuous delivery requires continuous testing, and you can’t have continuous testing if you don’t have automation.

Wayne: Very interesting. So, I couldn’t agree with you more. It seems like the time pressure associated with the cycle is increasing. But what about risk? I think, before, when I look back over the years of how testing has evolved, especially in agile environments, I saw agile environments take testing back to this “task type level”, right? Where it was part of the burndown, it became part of the story point, and people were asking the question, “Are we done testing?” And we seem to de-focus from this idea of “What are we really testing for?” So, where does risk come into the picture when speed becomes of the essence?

Jeff: Well, you’re trying to de-risk the whole process. That’s what testing’s all about. But nowadays, you can’t just have somebody focused on doing straight, pure testing. You’ve got to be thinking about the development angles in a sprint, and how you test in sprints, right, at the unit test level. A lot of times you see the developers and testers sometimes are the same person.

Wayne: Yeah.

Jeff: There’s a convergence of that development person and that testing person. Similarly, you’ve got out of sprint testing, you’ve got your system testing or your functional testing that goes on in parallel with that, but then you also have to think about the operations that support your development, your testing, and operations. That’s DevOps, right?

Wayne: Yeah.

Jeff: So we talk about quality engineering is the convergence of development, testing – which, test automation is the predominance, and then DevOps, into something called a quality engineer. So a quality engineer’s whole purpose in life is to de-risk the system that’s being developed, right?

Wayne: Okay.

Jeff: You talk about, you know, the whole … The testing bubble, which had this enormous amount of work that was taking place during the testing phase of the systems development lifecycle, and you had some stuff going on upstream that was your shift left, and stuff going on downstream that was your shift right.

Wayne: Yeah.

Jeff: And that’s how waterfall worked.Now, with an agile world, everything is happening at once. So we like to talk about something we call the “quality funnel.” Like, when you get a bunch of inputs at the beginning, and those inputs are requirements and new functionality and the funnel will close off towards the end, and you’re de-risking. You’re eliminating defects as you go, but you’re testing continuously.

It’s a very different mindset than saying “You’re doing your unit testing here, you’re doing your system testing here, you’re doing your user acceptance testing, your corner stats, etc.,” and on to the line of production. It doesn’t happen that way now.

Wayne: Agreed. So let me take one concept that you mentioned, and maybe expand on it just a bit. So this idea of dev-test or development testing – not quality engineering quite yet, but this idea of the developer actually doing more in terms of pulling their weight in terms of isolating or de-risking the process of creating code. What level of maturity do you think we’re at there, in your experience?

Jeff: You, Tricentis, or the world?

Wayne: Well, the world. Not Tricentis. You know, the world. Where’s the world at?

Jeff:  I think it’s low.

Wayne: Very low.

Jeff: Frankly. And here’s the reasoning. You still have a mindset out there, and one of the things I’m going to be talking about later on today is this diversity, and how/why diversity is important in testing. You still have developers out there that believe that they’re artists. They don’t want to test. And that’s okay. They come in with a mindset that they’re building something, but they still have to test what they’re doing.

But the quality engineers, as we talk about them, this new individual or this new entity, is going to have the skills to be able to do development and do testing at the same time, or you have a pool of people. So, if you pool quality engineers, and some people trend more towards the development in terms of their proficiencies, others will trend more towards the automated testing.

Wayne: Got it.

Jeff: And if you’re talking about Tricentis, the good news is that you can actually take today’s manual testers and convert them to automated testers.

Wayne: Absolutely.

Jeff: You can’t do that … Wolfgang Platz very accurately talked about SDETs this morning, and how you’re going to convert manual testers to be SDETs. He’s right. You can’t, but you can convert them to be automated testers. And then on the far side, you have people who are going to focus on DevOps, right, So it’s going to be a mix of skills that are going to create this quality engineering entity.

Wayne: So, with that in mind though, Jeff, let me kind of understand here. We’re not only maybe shifting this idea of where testing fits, but it’s also a rebalancing of that lifecycle as well, and distributing the tasks associated with the quality goals at specific stages of the funnel, if I can use the funnel analogy correctly. So, is that what quality engineering is, or would you give it a different definition than what I just tried to put in your words, or put in your mouth?

Jeff: No, I think that’s it. I think that’s a good description of what it is, and I like the way that we’re talking quality. Because, once again, let’s talk about eliminating the term “testing,” right?

Wayne: So that’s an interesting conversation as well because the term “quality” has always been so horrible to use. It seems to be an amorphous term that when you walk into an organization or a client, you can’t really describe quality, but you can describe the task of testing. So, I think over the years we’ve been forced to actually isolate the term into something that is tangible, right? Rather than “quality.” Now, the interesting thing is that in any discrete manufacturing process, the term quality is everything, right? So you know, you take an automobile shop, you take someone who’s producing hard goods, they have a quality process, and it always seemed unreasonable, over the years, how these things did not seem to merge together, especially from the idea of creating software, and you hit the nail on the head. It’s this creative element that organizations didn’t really understand.

Jeff: I would make one comment. In the past, testing has had a challenging reputation.

Wayne: Yeah.

Jeff: Some people don’t necessarily want to be testers, or to be in a testing world. So, we consciously, at Accenture, try to define our testing practice as something different, right? We are not testers. We are stewards of quality. Right? We’re protectors of our client’s production, protectors of Accenture’s business, essentially. So, we focus on quality, we focus on transformational quality. We don’t just have people sitting in back rooms doing test planning, scripting, and executing.

Wayne: Yes.

Jeff: Right? We’re getting away from all that now. You may have to do a little of that at the beginning of your career, but over time you’re going to evolve into something that’s … I like to use the word sexy, right? Testing can be sexy.

Wayne: I like to use it, too.

Jeff: Quality is sexy.

Wayne: It usually gets a good laugh at conferences.

Jeff: Exactly. Right. People will treat you like you have three heads.

Wayne: So. I have one last question for you, Jeff, and this is something that I think has raised its head at this conference. You know, not only are we seeing digital our businesses, but we’re also seeing digital hit the idea of testing. So, concepts like AI, AR, robotics, whatever you might mention, also applies to the actual quality process. I was about to say testing, but I will use the word quality. Quality process. In your opinion, what do you think the most exciting element is? What’s coming down the pike, in your mind, that you really have your eye on?

Jeff: It’s already here. We’re looking at using analytics.

Wayne: Got it.

Jeff: Using not just intelligent automation, but artificial intelligence to improve the way we do quality engineering. So, here’s an example: We have additional ecosystem partners where we can take a look at the basis of functionality that needs to be scrutinized – I won’t say tested, scrutinized. Validated, if you will.

And to determine how much testing do you need to do, and where do you need to spend most cycles on testing?

Wayne: Interesting.

Jeff: That has shown to eliminate about 15-20% of the testing that you need to do.

Wayne:  That’s … So just on the analytics alone?

Jeff: Just on the analytics.

Wayne:  Just working smarter.

Jeff: So here’s an example: If you talk about intelligent automation on the one side, in Tricentis, if you talk about analytics and artificial intelligence on the other side, we think the intelligent automation with Tricentis can yield a better than 50% cost savings. Why? Because we’re eliminating manual testing.

Wayne: Wow. Okay, so that’s pretty huge.

Jeff: It is! Right, so we … Traditionally we try to automate regression testing, right? Been somewhat successful. The UFTs, the Seleniums of the world, get you there but you got to maintain scripts.

Wayne: Yep.

Jeff: With Tricentis you don’t have to maintain scripts, right? It’s much easier to do so. So now, you can use it for not just doing regression testing, you can do it for progression testing. The system testing. So, you really don’t need to do manual testing. Or if you do, sorry, you do exploratory testing. Which Tricentis also does.

Wayne: Absolutely. So that’s a good point. You know, I think also what we’re also seeing in the evolution of the actual testing industry is more automation, and now the combination of better analytics has allowed us to actually push the envelope just a little bit more and get more done automatically.

Jeff: Yep.

Wayne:  So you know, what I always like to say is … I think ten years ago, Jeff, I think like … certainly 15 years ago, you know, the “term test automation” was here but it wasn’t keeping its promise. I think the industry actually had a lot of overhead associated with it. We actually you know, obviously, were baked-in waterfall … Agile was here, but we still were baked-in waterfall processes, which actually forced us into actually doing more work, and then you had to make a tradeoff decision. “Do I spend the time automating, or do I just get it done manually?” Right, and I think the means to the end was easier to just go knock off a bunch of manual tests in order to hit the deadline.

Jeff: Here’s why, though. It’s because back in those days, it was scripted automation testing. Nowadays we’re talking about non-scripted. No underlying code to the scripts you’ve created. They have a functionality you’re creating, right?

Wayne:  Yeah.

Jeff: That … The implication is huge. It means you don’t have to maintain a bunch of scripts that you put on a shelf.

Wayne: Exactly.

Jeff: Two years later, it’s going to be completely obsolete. Now you just tweak that model, right?

Wayne: It’s a term I like to call “bloat.”

Jeff: Bloat, yes. Exactly. That’s what’s enabling us to systematically eliminate the need for manual testing.

Wayne: Very nice.

Jeff: I’ve got to ship my work for us. I’ve got 40,000 people globally, right? And our competitors do as well. The 40,000 is going to shrink, right?

Wayne: Yeah.

Jeff: And now it’s going to be, instead of having a mix of predominantly 80% manual testers, we’re going to have 80% quality engineers that have skills that are in line with automated testing, DevOps, and agile development.

Wayne: I think you’re absolutely correct. It’s the wave of the future.

Jeff: It is.

Wayne:  I am here with Jeff Wilkinson. I’m not going to try your title again. I’m just going to call you managing director at Accenture. Jeff, thank you so much for spending the time with us, and I look forward to speaking with you again.

Jeff: Thank you, Wayne. Appreciate it.