bird image


The Promise of Test Automation

When was the last time you visited the LinkedIn group Software Testing & Quality Assurance (QA) @standqa? Stop in and you will find a barrage of posts about the challenges with Test Automation.  Clearly, there is plenty of interest in the subject, but painfully, very little success.  Posters are commonly asking for help with a particularly juicy problem, or asking for advice on methodology and tooling.  My contention is that this trend is due to the QA Software industry categorically failing to deliver on the promise of test automation.

First, a disclosure and a disclaimer.  I lead sales for one of those QA Software Industry firms, Tricentis, so if you believe that a salesperson is lying when their lips (or keyboard) is moving, stop here.  I’m not trying to sell you on our product, although I’d be happy if you took a look!  My goal is to raise awareness in an industry I feel is destined for another colossal let down.  The disclaimer is that I am not espousing that our tool, or ANY tool will solve problems with QA.  This requires a balanced approach of people, process and technology.  But over the years, as development methodologies have changed, from waterfall to iterative and agile, QA practices and processes have forced testers to compensate for a lack of viable tooling.  Essentially, the technology has not kept up with the pace of people and process.

Of course, software companies have tried to sell the latest and greatest toolset.  How many times have you heard the claim that test automation would improve your time to market and lower your cost?  “The Return on Investment is huge!”  Yeah, right.  Take an honest look at your environment.  What level of test automation can you achieve, and cost effectively maintain?  Gartner says it’s less than 20% on average.  In my experience, best in class is still less than 50%, with a significant investment in frameworks and well defined, often heavy handed process driving compliance.  But what about Agile teams?  Do you have a “hardening” sprint to fix all the issues that have been missed, or a n+ model to automate the mission critical test cases from some bygone sprint?  How can teams be truly agile when most testing is done manually, often off-shore?

As the industry has repeatedly failed to deliver on the promise of test automation, we have become jaded.  More activity is now spent with open source tools like Selenium and SoapUI, especially @standqa!  When faced with the realization that test automation tools do not deliver desired results, the next best option is to get “as good” for “much less” investment.

At Tricentis, we believe that the scripted approach is the central problem.  Writing code to test code is inherently flawed, fragile, time-consuming and expensive.  Our model is based on three simple concepts:

  • Know what to test, and test what contributes the most to business risk first
  • Automate as much as possible, as soon as possible
  • Virtualize environments and test data to ensure that tests execute, regardless of external variables

To achieve this model, we developed Tosca, not to replace people or process, but to provide a tool that can keep up with modern development. Whether you look at Tosca or not, remember that your development team is not slowing down.  With DevOps models, production is ready for the pace.  Where will QA be if we can’t keep up the quality and meet the ever increasing pace?

Want to know more? Read about Tosca Testsuite’s Model Based Test Automation here!