Agile testing

Artificial Intelligence, Communication, & the Evolution of Software Testing: Interview with Isabel Evans

At the StarEAST software testing conference in 2017, Isabel walked onto stage during the popular “Lightning Strikes the Keynotes” session, and delivered a presentation on ancient Scottish sheep farming practices. The talk, which could only last 5 minutes, was informative, witty, profound, and extremely relevant to software testing – all at the same time.

That is Isabel Evans for you. A highly-regarded speaker in the conference circuit and luminary in the software testing world, she approaches the challenges of Quality Assurance with deep insight. Combine that with her thirty years in the IT sector and you get a rare, tour-de-force perspective – one that can tackle the daily challenges faced in QA from a broader, “big picture” position.

If you want to see Isabel speak in person (and you really should!), check out her upcoming events and master classes here.

Chelsea: Can you give a quick overview of your background and areas of interest?

Isabel: I have been in the IT industry since the 1970s. I was a developer in the 70s, and moved into testing and quality assurance in the 80s. Then through the late 80s and 90s, I moved into quality management and began looking at organizational quality as opposed to just software quality.

Since then I have developed several interests. I am interested in user experience, UX design and testing, and about how you can take the engineering qualities in software and see how they merge into UX attributes like trust and flow.

Another area that I have worked in is organizational quality, where I explored organization excellence and improvement. IT is really just a tiny part of the picture – it is just a service into the rest of the organization. So how do we, as an organization, improve and build better things?

Within that, I developed an interest in team work and how people interrelate. That was partially driven by the fact that I am not particularly good at teamwork and networking myself. I realized that there are many other people in the IT industry who are not particularly comfortable with that either. It’s a bit of a stereotype, but sometimes those stereotypes are there because they fit.

So, I started looking at ways in which you can work with other people even if you are not necessarily comfortable doing that. A lot of the techniques companies can put into place (for example,  formal reviews) are about getting people to communicate, and giving them a structure within which to do it. How do you develop those structures so that they are flexible and encourage people to communicate in ways that are positive?

All of that has come together into my main interest at the moment: The UX and usability of testing tools for testers. There is a big focus in the industry right now on the fact that test tools are not necessarily easy to manipulate. For a while, people have been saying that testers need to become more technical so they can fit the demands of the tools they are trying to use and leverage them more easily.

Chelsea: I see what you mean. What would it look like if tools developed to fit the demands of the testers that are using them? What if the tools meet the testers where they are?

Isabel: Exactly. What needs to happen to the toolset to make it more usable? I have noticed that there is  research going on around the developer toolkit (saying that the developer toolkit isn’t easy to use). Suddenly we have this phenomenon where it is acknowledged that highly technical and intelligent developers are attempting to use tools that they don’t understand. For many of these tools, 80% of the features go unused because the developers simply don’t know how to use them.

So, it’s not an issue with testers only – it’s a whole UX problem within the software industry.

Chelsea: It often strikes me how much testing comes down to people. In my observation, technology is all about philosophy and psychology. We get caught up talking about the technical details, but at the end of the day, it is psychology that drives how we interact with those details. It is amazing how even when we are talking about machines, everything is so human driven.

Isabel: In the 70’s or 80’s someone famously wrote, “Don’t talk about computer interfaces – all interfaces are human interfaces.” The point was that even if you are talking about something that is embedded into a piece of firmware, humans had to create that interface, and they gave their interpretation to how it should work. That is even more true when it comes to the actual UX of a product.

Chelsea: So if we, as humans, are evolving, then testing must be evolving too. How have you seen testing change in your career?

Isabel: It is sort of going around in a circle. When I first started in the testing industry, there weren’t many specialized testers. I didn’t encounter specialized test groups for a while. When I first became involved in a specialized testing group, the focus was on exploring the software, and then translating those tests into automated tests. We did a lot of automated testing. Then, in another organization, through the 80s and 90s into the 2000s, there was a loss of that automation because it was perceived as too difficult to do. Suddenly an emphasis on manual scripting and technique arose. Now we come back into the 21st century, when people are starting to talk more about automation and exploratory testing, and less about manual scripting.

At the same time, development has gone from small focused teams working on a specific problem, through to big projects with silo working, and now coming back to people saying they need more frequent deliveries – essentially the rise of Agile and DevOps.

I find it quite fascinating that it feels like we are just cycling.

It seems to me like there is a tension between a desire to create in a completely free sandbox, and the need to control that space out of a desire to “become serious about it”. The problem is that if you control it too much, you damp down the creativity and people will eventually feel like they need to burst through into the “next thing” open sandbox. It’s a sort of evolution in how people deal with ideas.

With each of these stages we go through, there is an acceleration, a change in how we do it. People are talking about the 4th industrial automation revolution going on now. As a society, we have gone through 4 industrial revolutions, and each time people say this is the time that the machinery will overtake peoples’ craft skills, and everyone will lose their jobs.

Much of that fear now revolves around Artificial Intelligence, but the truth is that it still looks like we are nowhere near being able to relinquish the running of the world over to machines. At some point, if that arises, we may be in a new reality of perpetual leisure, but at the moment, all the automation we see requires human interaction or human direction. How long will that last?

When I was first in the industry, studying computer science, my final thesis was about whether machine intelligence would ever be possible. I focused on the subtopic of natural language processing. Back then, the idea that anything like Siri or Alexa would exist was laughable. It was not that long ago that people thought that kind of technology was simply impossible.

Chelsea: On a small scale, say the next 2-3 years, how much do you think artificial intelligence is actually going to supplement our testing?

Isabel: I wouldn’t like to make a call on that. When I hear people talking about that, it seems so close to the research and so far from what I see in most people’s experiences. While most people are struggling to do automation at all, artificial intelligence seems far off. And yet I have a feeling that there will be a break-through and it will suddenly flip – like smart phones. There was a long period when people didn’t think there would be a market for it, and then suddenly it becomes unthinkable to not have one. When artificial intelligence does become mainstream, I think it will be like a flip of a coin. Until that point, it will probably look quite unlikely to most people.

Chelsea: It seems to me that innovation works in a cyclical manner like testing – that you suddenly have this cloud-burst of brilliance and breakthrough in a certain place, that lasts for a period, and then there is a space, and another burst of ideas. It also seems that the space between those bursts of innovation and creativity has become so much shorter.

Isabel: When ideas move from being crazy to being mainstream, suddenly everyone can do them. Prior Roger Bannister’s running the 4-minute mile, people said that it was impossible. But once he had done it, lots of people began accomplishing the same thing. It’s because they suddenly realized that it IS possible, and removing that mental block allowed them to accomplish greater things.

It’s the same with things like DevOps and continuous delivery.

Chelsea: So, when it comes to DevOps and Continuous Delivery, while the tools and resources are important, a large part of the problem is actually that mental block.

Isabel: For any idea, technical or otherwise, you have early adopters. There is a chasm between the early adopters and the mainstream. Once the idea crosses that chasm, and more people are using it, it turns into the norm. On one end of the curve you have the people who just want the new ideas, regardless of whether it is buggy, and on the other end you have the people who are never going to change. But once the idea crosses the chasm, that change is quite rapid. With any new idea, the challenge is getting the idea over the chasm. When you see that change happen, you hear the language change from “this is new, this is untested, and dangerous” to “this is proven and everyone is doing it”.

Chelsea: Is this where you see Agile and DevOps being now?

Isabel: I think Agile has crossed the chasm as a concept, and DevOps is just doing that now. Agile is maybe a third of the way through the “innovation curve”. [Editor’s note: See Geoffrey Moore‘s work on “Crossing the Chasm” for more information on the innovation curve.]

A vast number of people are still doing waterfall. They are hearing about Agile and DevOps, and are getting worried that maybe they should be doing it, but they have not yet adopted it. It may not be appropriate for them, or they may not accept it culturally.

The funny thing is that some people who were the earliest adopters of Agile are now saying that it has morphed into something that is not what they intended. As it becomes more widely adopted, it becomes standardized, there are certificates associated with it – it’s not new and exciting anymore. They are pushing things in place to make it safer, which makes it less appealing to the early adopters who will begin looking for something new. Agile needs to change and adapt… to be agile.

So when we talk about that tension between the free sandbox and the need to be “serious” about doing something, we can identify the same tension in Agile and DevOps.

Chelsea: In regards to communication and teamwork – do you think Agile and DevOps has been improving that as of late, hindering it, or is it just the same-old all over again? 

Isabel: One of the things that is interesting about the way that Agile has been implemented in some places, is that you become forced to communicate with people on a frequent basis. There are lots of good things about that, but for some people, it is absolutely exhausting. So, does Agile and DevOps increase the frequency of communication? Yes. Does it improve the quality of that communication? Not necessarily. Does it make communication easier? For some people, yes, but for others it makes it more difficult.

When you set up an agile co-located team, you will certainly have shared spaces, but it is also beneficial to have “caves”. Once you go into your cave, you can be left alone and work on something without needing to communicate. That gives you the rest you need to be able to deal with things in a common space. Remembering those human factors is very important, because it is just as bad to force everyone to be in constant communication and interaction as it is to force people into silos.

Chelsea: Do you think that, when it comes to DevOps, communication is one of the biggest challenges to making it work?

Isabel: I think it is. DevOps is more than sitting the Development and the Operations team in the same room. And yet some organizations believe that is all it takes to “do DevOps”.

Here is another question you need to consider in regard to whether communication is improved or not. Suppose you have set requirements that are stable and very complex. So, the best way to communicate those requirements could be to write them down, and then study them very carefully. And if you are dealing with something that is stable, complicated, and you have the time to do it, a formal review might just be a better solution that attempting to understand the requirements via Agile story cards.

There can be a “baby and the bathwater” issue with this sort of thing. The principle of Agile is that you are flexible to change how you do things based on the project or requirements. Not all the practices of Waterfall are bad – sometimes they are the best approach, and it should be within the scope of Agile to realize and implement that appropriately. The challenge is recognizing what is most applicable to the project.

You don’t have to take a standard operation like Waterfall and make it bureaucratic for everyone. And you don’t have to take Agile and say that now we can never have documentation – you can blend it; you can empower people to make sensible choices for the situations that you are in. That is the ideal of Agile.

Isabel Evans Personal Bio:
Independent quality and testing consultant Isabel Evans has more than thirty years of IT experience in quality management and testing in the financial, communications, and software sectors. Her quality management work focuses on encouraging IT teams and customers to work together via flexible processes designed and tailored by the teams that use them. Isabel authored Achieving Software Quality Through Teamwork and chapters in Agile Testing: How to Succeed in an eXtreme Testing Environment; The Testing Practitioner; and Foundations of Software Testing. A popular speaker at software conferences worldwide, Isabel is a Chartered IT Professional and Fellow of the British Computer Society, and has been a member of software industry improvement working groups.