Agile testing

Testing Tools vs. Test Automation: Interview with James Bach, Part 3

If you have spent any time immersed in the world of software testing, you have probably come across James Bach. James is a noted thought leader, author, blogger, speaker, proponent and developer of exploratory testing, creator of the Rapid Software Testing ™ methodology, and all-around disrupter.

Read Part 1 of the interview, “The Essence of Testing”, here.
Read Part 2 of the interview, “Throwing Away ‘Bad Software Testing’ with Agile and RST”, here.

It is no secret that most organizations are feeling the pressure of increased testing demands, faster time to market, and increasingly high consumer expectations. For many of these pressured organizations, simply introducing more automation is a temptingly easy solution. If you ask James however, he’ll tell you that such a solution is bound to fail, and why we should be more focused on finding valuable “tools” than simple “automation”.

Chelsea: You are self admittedly a “tool skeptic”, and are outspoken about your feelings on automation. What is your perspective on test automation tools, and their impact on a manual tester?

James: I don’t, and have never seen testing as manual labor. Testing is a puzzle-solving process for me. Testing is a learning process. So when someone says, “won’t this tool make you more productive?”, I go, “tools have always made us more productive!” That’s not the issue.

The people in my tool-skeptic part of the testing world are not saying we don’t want tools or don’t think that they are useful – we are saying that our relationship to our tools (just as in the programming world), is that of “practitioner to heuristic”. Master to slave. I am always in charge.

So when I use a tool in my testing, I am constantly thinking, “is this tool actually helping me? How much will it help me, and when should I say stop?” So instead of just automating a test, I am thinking, “how can this tool help me?” which is a very different intellectual attitude.

Chelsea: In that case, it sounds like you are assuming certain stance when someone talks to you about automated testing.

James: That’s an interesting way to phrase it. Yes, I would say that I am assuming that based on their words and what is “normal” to me. This is where critical thinking comes in. Although it’s true that I do make such assumptions, and I would argue that they are productive assumptions, I must be ready to abandon them as the evidence comes in.

That’s part of being a good technical person or thinker at all – that you may make an assumption, but you are willing to revisit your assumptions in the light of new evidence. So yes, I am assuming a certain attitude, and that is because I see this attitude all over the place.

Chelsea: It does seem like a large portion of the testing industry does not have a human-centric viewpoint like yours. Often then, assuming that “automation” implies becoming a slave to a process or a tool, as opposed to viewing testing as an act of creative and critical thinking, is accurate.

James: I think the word automation itself is very telling. I use the word “tool”. When someone says “automation”, I know they are talking about a tool that does something, but to say “automation” instead of saying “tool” is to subtly demote the human operator. If I say “tool”, like a carpenter talks about tools, you always know that the carpenter is in charge of using the tool, choosing it, deciding when the tool is done, deciding if the tool is working, and evaluating the work, etc.

When you use the word “automation”, that suggests something like a washing machine, where you put your clothes in, set the timer, and walk away. There’s not a sense that you are operating the washing machine – more that the washing machine is doing it’s thing. Even though the washing machine is just a tool, it feels like automation.

So when people say “automation”, my hackles go up a bit because of the implication I seem to feel, that no human is involved. I would prefer to use the word “tool” to constantly remind ourselves that we are in charge of our tools. It’s easier to do that with words like “tool” rather than words like “automation”.

I am interested in tools that extend my powers. A tool that extends my powers might be a tool where I can give it a state model, and then say, “can you give me all the ways that this state model could be traversed and then put that into a file? And then could you give me 500 of those scenarios, chosen at random, for me to use in my testing? And could you tell me what percentage of that reflects my total coverage of the state’s base?” Those are the tools that are giving me a power that I don’t have. They are helping to understand, to penetrate into this complexity.

Ultimately what I look for is tools that help me generate new test ideas.