Exploratory testing

Why Exploratory Testing Isn’t Easy for Every Tester

Jan Jaap Cannegieter is a well-known consultant, author, (keynote) speaker, and requirements and test specialist from the Netherlands. He has worked as a tester, test manager, test consultant, and workshop leader. Follow him on Twitter: @jjcannegieter.

The popularity of exploratory testing is increasing. It’s used in more projects in which I’m involved in, more blogs and articles are written about it and more presentations at conferences are about exploratory testing. And I notice an increasing number of testers who apply exploratory testing, at least in the company where I work. And most of the testers love it! It provides more flexibility, freedom, and is more action focused. Often I hear testers say exploratory testing is, depending on the context, a better way to test software compared to detailed scripting. There is some scientific proof for this (Itkonen e.a, 2007, Afzal e.a., 2014, Allen e.a, 2012) but apart from the scientific proof the enthusiasm shown by testers for exploratory testing is also a kind of proof for me.

But this is not the whole story. Not every tester manages to adapt exploratory testing, and I have wondered why. I think it is important for testers to be able to apply different ways of testing like detailed scripting, global scripting, session based testing, bug hunts, test tours, and freestyle exploratory testing. A good senior tester or test manager should be able to determine which way of testing is best in a particular situation. So ideally, every tester and test manager knows how to covers the whole range with different ways of testing.

I started using different forms of exploratory testing (session based testing, bug hunts, test tours) instead of detailed scripting about six years ago and no, it wasn’t an easy or overnight change. By the time I tasted the benefits however, I was persuaded and didn’t want to go back to detailed scripting. What is it that stops other testers who stick to ‘scripting-no-matter-what’?

Are they simply loyal after years of experience with detailed scripting?

First I thought this was caused by the number of years someone had used detailed scripting. Most of the experienced testers I know were pretty successful using detailed scripting. On the other hand, it seemed that all the testing newbies within my company had no problem whatsoever with using different forms of exploratory testing. But some of the newbies, however, did prefer to make scripts in situations where their colleagues used exploratory testing. And then there was a whole bunch of testers with multiple years of experience in detailed scripting who embraced exploratory testing from the moment they tried it. I was one of the latter group – by the time I learned about exploratory testing I had more than 15 years of experience in scripted testing. And now I don’t want to go back. So the amount of experience a tester has with scripted testing is not what is holding some testers back.


So we are back to the first question: why is it so hard for some testers to embrace exploratory testing? I’ve come to the conclusion that the extent in which someone is able to apply exploratory testing is determined highly by his or her creativity. There are other aspects at stake here, but I do think creativity is one of the most important aspects. Testers who are less creative by nature will, in my experience, favor the guidance of scripts. The separation of different test activities (test planning, learning, test design, test execution, analysis of the outcome) will give them a sense of control over their test environment. For testers with a lot of creativity, scripting will create a feeling of restriction. By applying exploratory testing (the simultaneous learning, test design, test execution and the analysis of the outcome) they can fully exercise their creativity, turning them into advocates for exploratory testing.

Three types of testers

This theory fits neatly into the three types of testers I have observed:

  1. Testers who always (try to) apply scripted testing;
  2. Testers who always (try to) apply exploratory testing and
  3. Testers who can apply both and use the method of testing that they think is best in their situation.

The third type is, in my opinion, the best and most versatile tester. This type of tester can effectively test in different contexts. Does that make the first two types bad testers? That depends on the situation they are in. Let me sketch a context for the first kind of tester. That tester always wants to apply scripted testing –  if  he or she is testing a system that is best tested with scripted testing; then there’s no problem! But what if this very same tester tests a system that is best tested with exploratory testing? Then his or her skills do not meet the required approach in that context. This also counts vice versa for the tester that always favors exploratory testing. So testers who find it difficult to adapt exploratory testing should try to find a project or organization where scripted testing is the most preferable way. The same can be said for testers who prefer exploratory testing, they should try to find a project or organization where exploratory testing is the most preferable way.

Becoming more creative

As a tester, you should make sure that your testing preferences either meet the context you are in, or are flexible enough (read: creative enough) to adapt to your context. In my experience, that means a lot of us will have to become more creative. Creativity is not a static, unchangeable skill, it can, and should be, exercised and grown through practice.

Jan Jaap Cannegieter is a well-known consultant, author, (keynote) speaker in testing and requirements from the Netherlands. He has 20 years of experience in ICT and assignments in testing, quality assurance, TMMi, CMMI, SPI, Agile and requirements. Jan Jaap is currently test manager at the Dutch Normalization Institute and Principal Consultant at Squerist B.V., a company of  about 100 employees specialized in testing and process management. Jan Jaap is a thought leader, coaches other testers and test managers at Squerist and is responsible for product development. He is the driving force behind Situational Testing and he wrote several articles and books in the Netherlands.