When most people think of Agile testing, performance testers don’t usually come to mind. Despite the trend toward cross-functional teams, performance testers tend to remain siloed. Yet, you could make the case that exploratory testing–a key part of most Agile testing processes–was actually pioneered by performance testers. Much like exploratory testing, each performance tuning iteration starts with a defined question and an explanatory hypothesis that can be tested in a reproducible manner. Also like exploratory testing, it involves using the scientific method to correct and integrate previous knowledge via an iterative process. This process is essential to helping the tester measure, evaluate, and improve system performance. What can “regular” testers learn from performance testers’ interactive tuning approach? Our recent webinar with Ingo Philipp and Tim Koopmans, Optimizing Agile Testing: What Functional Testers Can Learn from Performance Testers, begins to answer this question. Ingo (a functional tester) shares what he has learned having watched Tim (a performance tester) in action. In his overview of exploratory testing, Ingo began by stressing that good testers are comfortable in the grey areas, because you will never know if you have tested every possible circumstance. There are time, resource, and budget constraints that mean that testers will only ever be able to check a subset of what can be known about any given piece of software. Testing can broadly be divided into confirmatory and exploratory testing.
- In confirmatory testing, testers want to achieve depth of knowledge. Confirmatory testing is done mechanically, with pre-defined data being put through pre-designed steps to confirm what is already known. This makes confirmatory tests great for detecting change, and well suited to automation.
- In exploratory testing, testers are trying to learn more about the product and expand the breadth of their knowledge by designing, executing and interpreting tests over multiple repetitions. As they learn new information, exploratory testers can detect potential risks and problems, something which requires a touch of human creativity.