be_ixf;ym_201911 d_22; ct_250
Software Testing Tool Comparison

White Papers

Software Testing Tool Comparison

Comparing Software Testing Tools

Gartner Magic Quadrant for Software Test Automation Joachim Herschmann, Thomas E. Murphy, Jim Scheibmeir November 27, 2018.

Gartner® Magic Quadrant for Software Test Automation evaluates 11 leading vendors. See how they compare.


The 2018 Gartner Magic Quadrant for Software Test Automation is now available. According to Gartner:

According to Gartner’s Magic Quadrant for Software Test Automation report, the software testing tools market involves tools that automate static code analysis, functional testing, and load/performance testing. “A functional software test automation tool enables an organization to design, develop, maintain, manage, execute and analyze automated functional tests for applications running on different platforms (including desktop, web and mobile). These tools can run automated functional tests by driving the user interface (UI) of an application (UI test automation), or interact with the application through an application programming interface (API test automation).”

Based on the criteria provided in the report, it seems that:
  • Gartner evaluated 11 software testing tools that met their criteria for enterprise-grade test automation tools.
  • The evaluation involved inquiries with Gartner clients, surveys of tool users, vendor responses to Gartner questions, and product evaluations.
  • Gartner required tools to support native Windows desktop application testing and Android or iOS testing support as well as support 3 of the following: responsive web applications, mobile applications, packaged applications, API/web services.
After reading the report, we believe that Gartner offers the following recommendations regarding software testing tools:
  • If you don’t already have a software testing tool deployed, start learning about the software testing tools market and evaluating tools that will meet your needs.
  • If you already use a software testing tool and are considering switching, consider adding complementary software testing tools to extend your testing capabilities without the hassle of switching.
  • If you’ve outgrown your legacy software testing tool, start considering alternative software testing tools based on your current and anticipated future (12 to 18 months out) needs.
  • If your existing software testing tool is meeting your needs, reassess the tool’s effectiveness at least yearly.

Download the complete “Magic Quadrant for Software Test Automation” report to review the results of Gartner’s tool comparison, their insights on Selenium and other open source software testing tools, and their detailed recommendations for test automation

Disclaimer: Gartner Magic Quadrant for Software Test Automation, Joachim Herschmann, Thomas E. Murphy, 27 November 2018. This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from Tricentis. Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.