How Forrester Ranks the Top API Testing Tools

Industry experts are now recommending that organizations place increased emphasis on testing at the API/message layer testing (services, message queues, database abstraction layers, etc.). They report that API testing is much more suitable for test automation and continuous testing than UI testing—particularly with Agile and DevOps.  Why?

  • Since APIs (the “transaction layer”) are considered the most stable interface to the system under test, API layer tests are less brittle and easier to maintain than UI tests.
  • API tests can be implemented and executed earlier in each sprint than UI tests.
  • API tests can often verify detailed “under-the-hood” functionality that lies beyond the scope of UI tests.
  • API tests are much faster to execute and are thus more suitable for checking whether each new build impacts the existing user experience.

Given these benefits of API testing, it’s not surprising that Forrester’s recent software testing research focused considerable attention on API testing tools. In the latest Forrester Wave on Functional Test Automation Tools, Diego Lo Giudice, Forrester Vice President and Principal Analyst, reports:

“As organizations shift to continuous testing, older approaches entirely focused on user interface (UI) testing become less effective. Going beyond the UI and testing APIs is crucial to avoiding brittle test suites and increasing test coverage. AD&D cannot achieve ruthless automation by focusing solely on test execution automation; it also requires automating test design and process orchestration.”

To help organizations starting or scaling their API testing efforts select the tool that best suits their needs, Forrester identified the top 11 tools that provide API testing as well as cross-browser testing, mobile testing, and UI testing. After evaluating these tools based on vendor interviews, product evaluations, and customer interviews, they scored the tools on 33 criteria and ranked them against one another. API testing capabilities were weighted the most heavily of all core testing capabilities:

“We evaluated each tool’s operating environment (including mobile, non-mobile, cloud, and browsers) and its design and execution automation; we placed the heaviest emphasis on API testing, some on UI testing, and to a lesser degree on packaged-app testing. We also evaluated automation execution and orchestration in continuous delivery pipelines, tool qualities, and third-party integrations with nonfunctional testing tools and Agile project management (PM) tools.”

API Testing tools covered in this report: Conformiq, Hewlett Packard Enterprise (HPE), IBM, LogiGear, Micro Focus, Microsoft, Original Software, SmartBear, TestPlant, and Tricentis