This is a guest article from Peter Kofler, the independent Code Cop. You can find the original version on his blog.
Developers need to be able to test.
I strongly believe that we (software developers) have to be able to test our work. How else can we make sure that “it works”? At the minimum we should be able to write unit tests for the code we just created.
The whole idea of Test Driven Development is based on finding, prioritizing, and running unit (and sometimes acceptance) tests against our code. Even people that do not like TDD put a strong focus on automated tests.
Paul Gerrard recently put it nicely, stating “if they can’t test they aren’t developers”. Unfortunately not all developers are able to do so. I often meet young developers who have no experience in deriving test cases from requirements or finding more than the most obvious test scenarios at all. They write tests because they want to make sure their programs work, but their tests are not focused, testing too much or nothing at all. More skill and experience is needed.
Do Testers need be able to code?
If developers need to be able to test, do testers need to be able to code? This is an old debate. As a hard-core coder I believe that testers need to be able to code. Period.
For example, at one of my previous employers, the testers were asked to automate their tests using Rational Functional Tester. They had to write Java code to drive the test engine. The code they wrote was abysmal. I do not blame them – nobody had taught them how to do it. But the code they wrote was important for the project’s success, so it better had some quality at the least.
What was needed were Test Automation Engineers. As the test automation code is much simpler than the regular code and there is no need for explicit conditionals or loops in test cases, I concede that these testers (the engineers) do not have to know as much about coding as developers do. They do not have to know Design Patterns or SOLID principles, but they need at least to know naming, duplication, and to a certain extent, abstraction.
Manual testing is immoral
Uncle Bob explained years ago why manual testing is a bad thing. Probably he meant that manual checking is immoral, because the testing community likes to distinguish between aspects of the testing process that machines can do (checking) versus those that only skilled humans can do (testing). There are more than 130 types of software testing, many of which can only be done by a skilled human, e.g. exploratory testing. I do not know how many testers do these kinds of interesting things, but most of what I see is either (immoral) manual checking or test automation engineering, which is kind of coding.
Austrian Testing Board
Earlier this year I was invited to the Austrian Testing Board to give a presentation about Deliberate Practice. While Deliberate Practice is popular with developers, e.g. Coding Dojos or Code Retreats, it can be applied to all kinds of learning. For example, there are Testing Dojos and Test automation (Code) Retreats. My presentation was received well and I enjoyed a heated discussion afterwards because – of course – I had provoked the “testers” a bit. 😉
One of the attendees, Gerd Weishaar, was surprised to hear about my approach: Doing TDD, automating all tests (because of “immorality”) and running only a few integrated tests (because of Test Pyramid and Scam). I also told him about my hatred for UI test automation tools because of their brittleness. As the VP Product Management at Tricentis, a company focusing on software (end-to-end) test automation, he had a very different view than mine: Almost no developers writing unit tests, groups of testers who work offshore, doing manual testing. We immediately became friends.
Gerd’s vision was to enable testers who are not able to code, empowering them to do their job – to test something – without any technical knowledge. Despite our contradicting views I was curious: Would it be possible to test, for example, a web application without any technical knowledge? How much technical background is necessary to understand the application at all? How much automation can be achieved without being able to write a script? And most interestingly, how much support can be put into a testing tool, just by making it usable for the testers.
How far can we go?
Tricentis Tosca Testsuite
The tool Gerd was talking about was Tosca Testsuite, a functional end-to-end testing tool. I had heard about it but never used it myself. Gerd encouraged me to have a look at it and provided me with online material, but I failed to allocate time for that. In the end he invited me to one of their monthly instructor-led trainings, free of charge. Although I had never planned to dedicate three full days (!) to checking out Tosca, I accepted his invitation, because I find well prepared instructor-led training an efficient way to get started with a new topic. The training was tough, three full days working the tool on various exercises. The trainer and the other participants knew a lot about testing and we had great discussions. (If you need to work with Tosca on your job I strongly encourage you to take their instructor-led training. Afterwards you will be productive using the Tosca Testsuite.)
Small Disclaimer: I attended the three day training free of charge. Gerd asked for feedback and a blog post about it in return. This is this blog post. I liked the trainer and in the end I liked Tosca, so I am positively biased. Still, all of the open questions from the beginning of the article remain.
I had two goals for the training: First I wanted to learn more about the mind-set of “classic testers”, and second I wanted to check Gerd’s idea of empowering non-technical/non-coder testers. Tosca is very powerful. It includes tools for project planning, risk management, and test case design as well as manual and automated testing. It took me three days to get an idea of what the tool is capable of. I do not know how much more time one would need without the development background that I have. I am used to such kind of tools – after all Eclipse or IDEA are not easy to use either. So training is necessary to use Tosca Testsuite. Maybe three days are enough to explain it to someone who has no idea about coding, maybe five days are needed. Anything less than ten years will do. As the tool only offers a fixed limit of combinations, it is possible to get familiar with it fast.
Tosca is a professional tool. The user interface is well done and it supports power users by providing tab ordering and consistent keyboard short-cuts. Unfortunately some of its many options and functions are hidden deep in dialogues or require you to type in certain text combinations. This is not usable in the general sense. I suppose developers are used to writing text, after all that is the code we write day after day. Still, we expect context sensitive help and code completion, something that non-coders need even more. So I guess there needs some work to be done in a few places, but that can be fixed easily, and Gerd told me they are planning to improve this.
Tosca Test Case Design
The coolest thing in Tosca is its Test Case Design functionality. It enables the formulation of abstract test cases, working with a happy path, boundary values, and negative cases, to create a conceptual model of all tests necessary to cover the given functionality. While this modelling does not involve any kind of coding related skills, it definitely requires skilled test professionals with analytical thinking and a high level of abstraction. As I consider these skills important coding skills, the test case design requires at least someone “like a coder”.
Tosca Test Automation
While it is possible to make test case organization or execution easy to use, the real problem is always the automation. I cannot imagine a non-coder being able to create reasonable test case automation. To hide complexity, Tosca’s automated test cases are constructed from building blocks which hide the real automation code, e.g. pressing a button on a web page.
It might be possible to create an automated test case just by arranging
How far can we go?
The underlying question: Is it possible to lower the requirements for technical work by providing smarter user interfaces that support the way we work? I hope so. Enabling people to automate tests without knowledge in coding is a good example for this. Even if we can make the tools just a bit more usable, our users will feel the difference. Tosca contains some interesting approaches regarding this. Reducing complexity lets users work on a higher level of abstraction.
Unfortunately abstractions are sometimes leaky and suddenly we are back in the low level world and have to type “magic” text into plain text fields. For now there are still areas in Tosca where you need to be a coder and you have to stick to software development principles to get good results. But I like Gerd’s vision and I am curious how far they will go to empower non-technical testers.
Read more about test automation here!