AI-powered but not autonomous yet: How artificial intelligence can address three common UI testing challenges
Learn how AI can be used to eliminate common testing headaches, as well as what the future of autonomous testing holds.
Today we’re faced with both a growing number of ways to test new software and a growing number of tools to accomplish those tests. And who really has time to keep track of the hundreds (and counting) of tools out there? Fortunately for you, we do. Use the navigation to the right to jump to different testing tool categories.
Before you get lost in the deep indexes of Google trying to figure out the best software testing tools for your needs, take a look at this trusty list developed by the software testing experts here at Tricentis. Be sure to check out our software testing tools after spending some time here.
Here’s what you can expect:
There are 100+ software testing tools and 8 categories all with the following information:
We’ve broken down our recommendations into 8 categories, plus a bonus section with up and coming tools to keep an eye on:
Test automation is one of the most mature software testing segments. Currently, the rise of good open source options and the push toward DevOps and multichannel models are transforming this segment.
We can break down test automation tools into two subcategories:
Why use Functional Testing tools? Functional Testing tools drive automated tests by interacting with the UI layer of an application. These tools usually provide a record and playback interface, making it easier for nontechnical testers to provide automation coverage. However, these tests can be slow and brittle.
Summary: Provides model-based test automation focused on test case design versus problem solving. Tricentis Tosca also offers end-to-end testing and recording capabilities.
Features:
What you really need to know: With the Tosca Testsuite, we have focused on creating model-based automation and a “mini-suite” that goes beyond pure test automation to provide risk coverage and test case design as well.
For more information: Check out our site.
Reviews: Head over to ITCentralStation.com or G2Crowd.com to read reviews.
Summary: Previously known as QuickTest Professional (QTP), HP UFT (Now Micro Focus UFT) automates GUI functionality and back-end service testing. It also offers reusable test components, helps convert manual testing assets into automated ones, embraces the shift left to make testing part of the Agile development cycle, and allows for testing across multiple machines and devices.
Features:
What you really need to know: Although HP remains the market share leader in paid test automation, it has recently struggled to focus on newer platforms, ceding ground to competitors.
For more information: Check out the Micro Focus UFT site
Reviews: Checkout UFT reviews over at ITCentralStation.com
Summary: Part of IBM’s larger Rational testing platform, the Rational Functional Tester uses data-driven testing to automate functional and regression testing. It supports a variety of applications and allows for both storyboard testing and test scripting.
Features:
What you really need to know: Once a true market leader, IBM has had to focus on standing its ground against rising competition from HP and up and coming players in recent years.
For more information: Check out the IBM Rational Functional Testing site.
Reviews: Checkout a few reviews over at TrustRadius.com and ITCentralStation.com.
Pricing: Plans start at $3,400 per month with 6 pricing tiers. Visit their pricing page to learn more.
Summary: A complete web and mobile automation framework built on top of Selenium and Appium, Katalon Studio is packed with essential features that only available in commercial tools while remaining as a free solution.
Features:
What you really need to know: Katalon Studio is a cross-platform tool to transform your manual testing into automation that requires minimum program background.
For more information: Check out the Katalon Studio site.
Reviews: Head over to Capterra and SoftwareTestingHelp to read reviews.
Pricing: Free
Summary: Automates functional testing of business processes across the entire application landscape. Worksoft Certify offers capabilities around discovery, validation, measurement, compliance and scale.
Features:
What you really need to know: Although Certify works with a wide variety of applications and custom built code/extensions, it’s focus is on supporting package software like SAP, Oracle, Salesforce, etc.
For more information: Checkout Worksoft Certify automated business process testing.
Reviews: Checkout reviews at ITCentralStation.com.
Summary: An automated functional testing tool that uses an image-based approach, which looks at the screen of any device the same way a user would. This image-based UI testing differs from the traditional approach because it tests from the perspective of a user rather than by looking at code. Eggplant Functional works across all device types.
Features:
What you really need to know: eggPlant Functional’s use of an image-based approach to automation can help solve complex automation problems. Their approach allows testers with little to no automation skills to achieve robust automation overage and for teams that have struggled to implement automation, its a way to get over the hump.
For more information: See eggPlant Functional’s download page.
Reviews: Head over to ITCentralStation.com to read a couple reviews.
Summary: Allows for setup and execution of test automation scripts and offers GUI object recognition, reusable code modules and record/play back functionality. Ranorex also offers script-free functionality.
Features:
What you really need to know: Ranorex provides similar functionality to integrated suite vendors with lower cost licenses and maintenance.
For more information: Checkout the Ranorex site.
Reviews: G2Crowd.com and ITCentralStation.com have a good amount of reviews.
Pricing: Plans start at 2,990 USD per license with 3 pricing tiers. Visit their pricing page to learn more.
Integration/API testing tools send various types of payloads through APIs to make sure they accept valid inputs and reject improper ones with proper messages.
Some of these software testing tools can also provide testing for load. Integration/API testing is especially important for cloud, mobile and PaaS. Let’s look at a handful of Integration/API testing tools to consider:
Summary: Uses a declarative workflow model that focuses on building tests over scripting. CA Application Test (now part of the BroadCom Continuous Testing solution) embraces the shift left by allowing for API testing before UI development. It also offers automated mobile testing, visual tests, customized load testing and advanced analytics.
Features:
What you really need to know: CA Technologies is mostly focused on competing in the mainframe space and testing desktop applications and complex infrastructure using LISA.
For more information: Checkout the Broadcom Continuous Testing website.
Summary: A comprehensive test automation tool with integration testing capabilities as well as mobile, regression, performance and scalability testing capabilities. In terms of integration testing, IBM Rational Test Workbench allows for service-level testing, automatic scheduling and execution of testing via an integration with the IBM Rational collaborative lifecycle management tool.
Features:
What you really need to know: IBM Rational Test Workbench provides a complete set of enterprise-ready testing tools that cover testing needs throughout the development lifecycle.
For more information: Checkout the Rational Test Workbench automation solution for load and integration testing.
Summary: Provides test automation for end-to-end scenarios across multiple endpoints with support for REST, web services and over 120 protocols/message types. Parasoft SOAtest creates extensible and reusable tests. In addition to API testing, it also offers SOA testing, web and performance testing, web UI testing, runtime error testing, API security testing, service virtualization, and development testing.
Features:
What you really need to know: Parasoft SOAtest offers automated API testing as well as several other testing capabilities. Its most unique feature is the extensive list of environments, including protocols, platforms and messaging formats, that it supports.
For more information: Checkout Parasofts SOAtest API testing solution.
Reviews: Read a few reviews of SOAtest at ITCentralStation.com
Summary: A complete API testing platform with support for API functional testing, API load testing, API security testing, service virtualization, API testing in code, API performance management and defining, building, and managing APIS. SmartBear Ready! API provides project management, metrics and reporting, script support, discovery, and continuous integration across all of these API testing capabilities.
Features:
What you really need to know: SmartBear ReadyAPI is a comprehensive API testing tool that covers API testing across functional, performance and post-deployment testing. It also integrates with the larger SmartBear testing environment.
For more information: Checkout SmartBears ReadyAPI testing tool.
Summary: Provides API service testing, API emulation and virtualization and API security gateway technologies. Crosscheck Networks uses a dynamic mutation technology to deliver functional automation, performance and security testing. It also offers API simulated behavior modeling with support for a variety of protocols and messages.
Features:
What you really need to know: Crosscheck Networks SOAPSonar offers a variety of API testing capabilities, although it emphasizes functional, performance and security testing for service endpoints. It runs these tests using a patented mutation technology.
For more information: Checkout Crosscheck Service Testing.
There are also several open source options for integration and API testing. If you decide to go the open source route, consider these tools:
Tool | Capability |
Apache JMeter | API Testing |
SoapUI | API Testing |
PowerMock | Object Mocking |
EasyMock | Object Mocking |
REST Assured | API Testing |
WebInject | API Testing |
Mockito | Object Mocking |
Jmockit | Object Mocking |
mocktail | Object Mocking |
WireMock | Object Mocking |
Betamax | Object Mocking |
Mockserver | Object Mocking |
Performance testing tools take a simple automation script and run it through hundreds or thousands of machines to simulate how an application will perform under load.
These tests are typically expensive and are run less frequently than functional automation tests. Performance testing is particularly important for cloud applications and there are two main types to consider:
This list represents both pure play and extension tools, each of which has its pros and cons. Here’s what you need to know about the difference between the two types of tools:
Pros:
Cons:
Pros:
Cons:
Tricentis Flood is a cloud-based, distributed load testing platform that helps teams test how their applications scale with massive load generated from around the world. Flood can generate load using popular open source load testing tools (such as Apache JMeter, Flood Element, Gatling and Chrome) as well as Tricentis Tosca functional testing cases (API and UI).
Flood is easily integrated with CI tools like Jenkins and Bamboo, as well as the most popular APM tools like New Relic, AppDynamics, and Dynatrace. Additionally, Flood can be integrated into your own private hosting account with providers like AWS and Azure.
Flood’s free trial comes with 5 node hours that can be used to jump-start your load testing. Subscriptions come in flexible options to meet your budget, with different options of testing capacity, features, hosting, and term length. Try it free here.
Summary: Automates any type of testing and offers cross platform support for mobile, web and desktop. Automation Anywhere can convert scripts into load scripts and allows for machine learning and analytics.
Features:
Tool Type: Extension
What you really need to know: Automation Anywhere is a lower end competitor to HP that offers some test case management capabilities as well as more robust automation capabilities.
For more information: Check out their automation solution.
Reviews: Head over to G2Crowd.com or ITCentralStation.com to read a few reviews.
Summary: Runs open source-based performance tests as well as scalable load testing. BlazeMeter is compatible with Apache JMeter, offers real-time reporting and mobile capture, simulates network conditions and allows developers to create and edit performance tests.
Features:
Tool Type: Pure Play
What you really need to know: BlazeMeter is a growing player in the cloud performance testing space. Although it has limited use cases and does not offer monitoring or post-production capabilities, it is popular with developers.
For more information: Checkout the BlazeMeter performance testing solution.
Summary: Offers unlimited scalability to simulate any size load as well as a variety of network technologies, including mobile. Silk Performer also provides visual diagnostics and stress testing with pattern detection. As part of Borland’s larger Silk Central platform, Silk Performer also includes a wide variety of supporting capabilities.
Features:
Tool Type: Extension
What you really need to know: Borland is a robust competitor to HP (although it does offer similarly high prices), with strong cross-platform support and cloud load capabilities.
For more information: Checkout Silk Performer load and stress performance testing.
Reviews: Head over to ITCentralStation.com to read a few reviews.
Tool Type: Extension
What you really need to know: With both LoadRunner and Performance Center, HP was traditionally a leader in this space, particularly for testing legacy enterprise applications. Micro Focus acquired these tools and rebranded under theLoadRunner family.
For more information: Check out the LoadRunner site.
Reviews: Read LoadRunner reviews here.
Summary: Offers code-free testing and real-time reporting as well as root cause analysis tools. IBM Rational Performance Tester also compares test results to SLA targets, integrates with IBM Rational Quality Manager and includes a test recorder.
Features:
Tool Type: Extension
What you really need to know: IBM Rational Performance Tester is cloud enabled and offers web and desktop support, but its mobile support is extremely weak.
For more information: Checkout the Rational performance test automation solution.
Reviews: Head over to ITCentralStation.com to read reviews.
Summary: NeoLoad uses an intuitive GUI for use case creation, target identification and test execution. It allows you to simulate load locally or using the Neotys Cloud Platform, supports web and mobile and monitors all servers during load testing.
Features:
Tool Type: Pure Play
What you really need to know: Neotys is a small but fast-growing player with cross-platform support for web and mobile (via NeoLoad) and post-production monitoring capabilities (via NeoSense).
For more information: Visit Neotys.com to learn more.
Reviews: Visit ITCentralStation.com to read reviews.
Summary: Provides performance, stress and concurrency testing with the ability to simulate various loads and test from different locations. Parasoft Load Test can also enforce Quality of Service metrics and run tests using tests previously created in Parasoft.
Features:
Tool Type: Extension
What you really need to know: While Parasoft Load Test does allow for reuse of test cases under load and provides performance monitoring and desktop and web support, it does not have any mobile capabilities and is not cloud enabled.
For more information: Learn more about Parasoft load testing.
Reviews: Visit the Parasoft review page on TrustRaidus.com and ITCentral Station.com
Summary: Simulates a variety of load conditions for an unlimited number of users and distributes the load across any number of machines. Radview WebLOAD can also monitor server performance and offers cross-platform support for web, desktop and mobile (via PerfectoMobile).
Features:
Tool Type: Pure Play
What you really need to know: Radview is a fledgling player in performance testing that was previously publicly listed but has since been delisted and is not performing very well currently (it has scaled down from 100 employees to about 25).
For more information: Checkout Radview performance testing
Reviews: Visit G2Crowd.com to read Radview reviews.
Pricing: Request pricing here.
Summary: A desktop tool for load, stress and scalability testing that works with both HTML sites and rich internet applications. SmartBear LoadComplete can generate load from virtual machines, on premise computers or the cloud and offers record and replay capabilities, visual programming, a test cost simulator, distributed load testing, analytics and functional test reuse.
Features:
Tool Type: Extension
What you really need to know: LoadComplete is a core SmartBear product and extension of TestComplete. It offers web support, is cloud enabled and provides monitoring capabilities, but it has no mobile capabilities.
For more information: Learn more about LoadComplete performance testing.
Pricing: Offers a free version with plans start at $230 per year and 5 pricing tiers. View pricing here.
Summary: Offers continuous load testing at any scale with real-time analytics. Akami CloudTest also has embedded user monitoring capabilities and allows users to build tests using RUM-based data and application-specific scenarios.
Features:
Tool Type: Pure Play
What you really need to know: Soasta is the largest player in the pure play space, with more than $75 million in funding to date. It provides cross-platform support with CloudTest for web and TouchTest for mobile (which also provides mobile automation) and offers post-production performance monitoring with mPulse.
For more information: Learn more about CloudTest load and performance testing.
Reviews: Read CloudTest reviews on ITCentralStation.com
Pricing: Offers a free version with plans start at $2500 per year and 2 pricing tiers. View pricing here.
Summary: Provides cloud-based, distributed load simulations with dynamic test controls and standard scripting languages. It also provides application-level virtual users for more complete test coverage. TestPlant’s eggPlant Performance tool is part of the larger eggPlant range of testing tools.
Features:
Tool Type: Extension
What you really need to know: eggPlant Performance can support web, desktop and mobile testing as an extension of eggPlant Functional.
For more information: Learn more about eggplant Performance here.
Pricing: No prices listed, but view licensing and pricing information here.
Test management is a mature testing segment currently being transformed by the widespread adoption of Agile, DevOps and Test Driven development. Most test management tools are either geared toward Agile or Waterfall development methodologies.
Below are Test management tools to consider:
Summary: Provides tracking, management, organization and reporting capabilities for software testing and test case management. qTest Manager sets up and manages requirements, organizes and manages test cases, executes tests, tracks defects and reports on test data. It also integrates with Jira, Rally and VersionOne as well as other popular automation tools.
What you really need to know: qTest Manager is a market leader in test management tools and is the only platform tool that offers a single source of truth for manual, exploratory and automated testing. It also offers the most comprehensive view of testing efforts, reporting and provides seamless integrations with leading automation tools and JIRA.
For more information: Learn more about qTest Manager here.
Reviews: Read qTest reviews on their G2Crowd.com listing page.
Summary: Provides visibility into the testing process with capabilities to manage, organize and report on tests. SmartBear QAComplete offers out-of-the-box templates or custom workflow options, defect logging, the ability to trace tests to user stories and reusability across the testing cycle. It also integrates with tools like Jira, Selenium and SoapUI.
What you really need to know: SmartBear QAComplete provides a single location to manage testing. Through customization, it can work for both Agile and Waterfall development processes.
For more information: Learn more about QAComplete.
Summary: Supports creation, planning, execution and tracking of tests. Zephyr supports structured and free-form tests as well as manual and automated tests and integrates with leading tools like JIRA, Confluence and Bugzilla. SmartBear acquired Zephyr in 2018.
What you really need to know: Zephyr is another player in the test management space, with an emphasis on real-time, on-demand testing. It is particularly focused on the smaller to mid markets segment through Atlassian, but is not built for the Enterprise.
For more information: Learn more about this test management tool at GetZephyr.com.
Pricing: Read how to buy Zephyr here.
Summary: A SaaS test management tool with both web and desktop clients that support various testing methodologies and offer management capabilities throughout the entire testing lifecycle. Testuff also supports automated testing, two-way integrations with bug trackers and video tracking of defects.
Features:
What you really need to know: Testuff is a SaaS tool whose most unique feature is its video defect tracking. It also offers a multitude of integrations with bug trackers and several integrations with automation tools.
For more information: You can learn more at Testuff.com
Pricing: Plans start at $27 per user per month with 3 pricing tiers. View pricing here.
There are also several open source options for test management. If you decide to go the open source route, consider these tools:
Tool Name |
Trantula |
Testopia |
QABook |
TestLink |
XQual |
TestMaster |
Testitool |
Named based on the shared suffix of its subcategories (security, usability, compatibility), -ity testing is more of an emerging segment featuring typically expensive and specialized tools. These tools are evolving, becoming easier to use and offering improved service, to assist smaller teams who cannot yet invest in dedicated resources.
We can break down -ity testing tools into three subcategories:
Security testing tools are typically built around a particular platform and/or technology. There is little open source pressure for security testing, so these are usually expensive, specialized tools that include heavy services. Here are the top security testing tools to consider:
Summary: Provides application security as a service with a single platform to view and manage security risk, develop security testing schedules and run remediation projects. Fortify on Demand runs automated tests with a full audit of results and includes support for the SAST, DAST and IAST spaces (due to addition of the legacy WebInspect tool) as well as limited support for MAST.
Features:
What you really need to know: HP is undoubtedly the largest player in the security space, with tools for SAST, DAST, IAST and MAST.
For more information: To learn more, visit the Fortify website.
Reviews: Visit ITCentralStation.com for a list of reviews.
Pricing: Plans start at $2000 per application with 4 pricing tiers. Request pricing here.
Summary: Veracode offers tools for SAST, DAST, IAST and MAST. Its binary static analysis is built toward Agile, while its DAST and IAST tools offer true cloud hosting and web perimeter monitoring. Additionally, almost all of its offerings are targeted to mobile.
Features:
What you really need to know: Veracode is the largest player in the security space without its own testing and SDLC tools.
For more information: Learn more at Veracode.com.
Reviews: Head over to Gartner.com to read reviews for Veracode.
Summary: Offers a single console for testing, reporting and policies and automates the correlation of static, dynamic and interactive security testing. IBM’s AppScan static testing tool is well known in the security space, although its DAST tools rely on third party solutions and its IAST tools only work with .Net and Java.
Features:
What you really need to know: IBM competes with HP as a similar full service offering, with some focus on downstream security tools like SIEM.
For more information: Head over to IBMs website to learn more.
Reviews: Visit ITCentralStation.com or Gartner.com for reviews.
Pricing: Plans start at $10,700 per install. Request pricing here.
Usability Testing Tools
Usability testing tools tend to be more basic and offer easy deployment within applications. The value of usability tools generally lies in the expertise or community provided. Take a look at the top usability software testing tools to consider:
Summary: Creates videos of people using a website or application and delivers a report that calls out problems and answers to any user surveys. UserTesting pools users from its own network or your company’s that match your target audience.
Features:
What you really need to know: UserTesting focuses solely on usability testing for websites and mobile apps. Its network of users and experts is strong and includes dedicated UX professionals. Although the product has traditionally been weak, it is improving.
For more information: Visit the UserTesting website to learn more.
Pricing: Pricing starts at $99 per video with 2 pricing tiers. View pricing here.
Summary: Allows you test live websites, mock HTML, images or prototypes and analyze the results via path reports and videos automatically flagged at issue points. Validately makes it easy to test on your own customers or find testers from its network and enables you to talk to these testers live.
Features:
What you really need to know: Validately focuses on user feedback before coding takes place. It is designed for users to interact with and provide feedback on desktop, web and mobile wireframes (not apps).
For more information: Visit Validately.com to learn more.
Pricing: Pricing starts at $79 per month with 3 pricing tiers. View pricing here.
Compatibility testing provides value based on efficient cross-browser mobile and web testing. Compatibility testing is simple to conduct manually, but is time consuming, expensive and risk prone. Technical or crowd automated tools bring efficiency to this process. Checkout the top compatibility testing tools to consider:
Summary: Provides a detailed report of layout problems to avoid combing through screenshots, locates Javascript errors and tests dynamic pages. Browsera can also test pages that require a login and test entire sites at once, eliminating the need to go page by page.
Features:
What you really need to know: Browsera offers both free and enterprise versions of its compatibility testing tool. Its most unique features include the ability to detect layout differences across browsers and to identify Javascript errors.
For more information: Learn more on Broswera.com.
Summary: Provides live interactive sessions for cross-browser testing with responsive testing and SSH tunnels for local testing. Browslering also offers screenshots, screen sharing, a bug hunter and videos. It is for web testing only.
Features:
What you really need to know: Founded in 2011, Browserling focuses only on web and does not support automated testing.
For more information: Learn more at Browserling.com
Pricing: There is a free plan, but pricing starts at $49 per month with 3 pricing tiers. View pricing here.
Test setup and infrastructure is a rapidly changing segment due to trends in mobile, web (cloud), NoSQL and containers, with many new companies emerging.
We can break down test setup and infrastructure tools into four subcategories:
Test cloud tools provide a network of virtual machines where desktop and web testing can occur at a lower cost, saving organizations money. Containers and AWS/cloud providers are lowering the barrier to entry for this market. Below are some of the top Test Cloud tools to consider:
Summary: Provides cross-browser, mobile and manual testing options as well as automation to test both web and mobile in parallel. Sauce Labs uses tunnel technology to encrypt traffic between individual tests and its cloud and offers access to live remote desktop sessions during tests.
Features:
What you really need to know: Sauce Labs offers everything as one product with additional enterprise capabilities in enhanced subscriptions. It currently only works with open source technologies like Selenium, Appium and JS Unit Testing. The additional manual testing and cross-browser comparison features it offers are very nice to have.
For more information: Learn more at Sauce Labs
Reviews: Visit G2Crowd.com or ITCentralStation.com to read reviews about SauceLabs.
Pricing: Pricing starts at $19 per month with 6 pricing tiers. View pricing here.
Summary: A SaaS load and performance tool for planning, executing and scaling web and mobile testing. HP StormRunner Load provides anomaly detection and real-time predictive analytics. It also supports the use of real mobile devices in cloud performance testing as well as test recordings.
Features:
What you really need to know: HP StormRunner Load is a highly scalable solution for Agile cloud testing.
For more information: View the HP StormRunner Load site.
Reviews: Head over to ITCentralStation.com to read reviews about StormRunner Load.
Pricing: LoadRunner/StormRunner Load is priced at $.56 per virtual user per day, but they have a free edition as well. See their pricing here.
Summary: A self-service platform designed to simplify the assessment, planning, designing and implementation of development and testing private cloud environments. IBM Smart Business Development and Test Cloud also offers on demand provisioning of physical and virtualized test resources.
Features:
What you really need to know: IBM’s Smart Business Development and Test Cloud is more focused on creating test environments than it is on creating machines/browsers from which to test.
For more information: Learn more about IBMs Smart Business Development tool.
Summary: Offers pre-configured cross-browser test environments in the cloud and allows for reusable test scripts. Silk Central Connect also provides side-by-side results of cross-browser testing for easy comparison and integrates with Silk Test.
Features:
What you really need to know: Silk Central is a more enterprise-grade version of Sauce Labs. Like Sauce Labs, it offers mobile support via emulators. However, it only runs Silk Test scripts.
For more information: Learn more about Silk Central Connect.
Reviews: Head over to G2Crowd.com to read Silk Central reviews.
Summary: Broadcasts a live RTMP stream of a browser to enable testing in real-time. BrowserTap offers parallel views to test multiple browsers at once and allows for testing of responsive design.
Features:
What you really need to know: BrowserTap has a long list of useful features it intends to implement, however it is still in the early stages of doing so and has a long way to go.
For more information: Learn more about BroswerTap.
Mobile device farms provide a network of mobile devices for testing to occur, saving device acquisition, management and maintenance costs. A recent entrance by Amazon with a fixed price offering has shaken up the market.
Mobile device farms to consider include Pure Play vs. Extension Tools. The list below represents both pure play and extension tools, each of which has its pros and cons. Here’s what you need to know about the difference between the two types of tools:
Pros:
Cons:
Pros:
Cons:
Summary: Provides automated testing against real mobile devices in the AWS cloud as well as remote access. AWS Device Farm also allows for manual reproduction of issues, integrations with development environments and use of built-in test suites, open source frameworks or manual tests.
Features:
Tool Type: Pure Play
What you really need to know: Amazon released Device Farm as an extension of Amazon Web Services in July 2015. Later in 2015, it release support for an internal scriptless UI automation framework in addition to existing Calabash, Appium and Espresso integrations. This framework is not yet popular, as most users are still using the open source integrations.
For more information: Visit the Device Farm.
Reviews: Head over to G2crowd.com for reviews.
Pricing: Plans start at $.17 per min with 2 pricing tiers. View pricing here.
Summary: Provides a management hub designed for continuous delivery workflows. Silk Mobile Testing also supports cross-platform automation tests, supports manual or exploratory testing and provides screenshots, videos and status reports from tests. It also integrates with Silk Performer and Silk Central.
Features:
Tool Type: Extension
What you really need to know: Micro Focus acquired this tool from Borland. While competitive at the time of acquisition, Micro Focus tools tend to be costly and not as frequently updated as competitors.
For more information: Check out the Silk Test website for more information.
Summary: Offers device inventory, application, user and project management for mobile, enables remote access and integrates with performance testing tools like LoadRunner and JMeter. Experitest offers both an onsite and a hosted, online version of SeeTestCloud.
Features:
Tool Type: Extension
What you really need to know: Experitest SeeTestCloud is an onsite device cloud that hooks into your available mobile devices. Experitest also offers SeeTestAutomation, which integrates with HP UFT and LoadRunner as well as some open source tools like JMeter.
For more information: Learn more about SeeTestCloud here.
Pricing: Plans start at $2000 per year with 2 pricing tiers. View pricing here.
Summary: Provides both real devices and emulators that support app testing, live monitoring and user insight delivery. HP Mobile Center also enables device management and user permissions, recording and replaying of interactions, structured manual testing or automated testing and security assessments.
Features:
Tool Type: Extension
What you really need to know: HP Mobile Center integrates with other HP testing tools to provide a more complete, end-to-end testing experience. Its support for both real devices and emulators provides flexibility in testing options.
For more information: Visit the HP Mobile Center site page for more information.
Reviews: Head over to ITCentralstation.com to read reviews.
Pricing: Plans start at $12,000 per year with 2 pricing tiers. View pricing here.
Summary: Provides test automation for real mobile devices. Keynote Mobile Testing supports unit, build acceptance, regression, explorative and negative testing. It also allows for test creation via scripting or integrations with open source frameworks and integrates with several CI applications.
Features:
Tool Type: Extension
What you really need to know: Keynote Mobile Testing allows for automated testing across a wide variety of devices and supports a multitude of integrations that allow the tool to work seamlessly with an Agile development process.
For more information: Learn more at the Keynote website.
Pricing: Plans start at $180 per mo with 3 pricing tiers. View pricing here.
Summary: Provides remote device management, tracking and monitoring as well as user and application management. Mobile Labs deviceConnect also offers manual and automated application testing, all from its private cloud testing platform.
Features:
Tool Type: Pure Play
What you really need to know: Mobile Labs deviceConnect is a private cloud-based mobile cart that allows users to connect devices they own to a private cloud for functional, automated or performance testing. Mobile Labs partners exclusively with HP for automated testing.
For more information: Learn more at the deviceConnect product page.
Summary: Offers cloud-based, automated testing as well as usability and performance monitoring. Perfecto’s Mobile Monitoring solution also offers visual reports and integrations with various automation frameworks, including open source options and HP UFT.
Features:
Tool Type: Extension
What you really need to know: Perfecto Mobile offers a combination of privately-hosted and shared/cloud-hosted devices.
For more information: Learn more at the Perfecto Mobile website.
Reviews: Head over to G2Crowd.com & TrustTradius.com to read reviews.
Pricing: Plans start at $99 per month with 3 pricing tiers. Request pricing here.
Summary: Provides automated testing for emulators, simulators and real devices for native, hybrid and mobile web tests. Sauce Labs is based on open source standards and works with both iOS and Android devices.
Features:
Tool Type: Pure Play
What you really need to know: Sauce Labs offers everything as one product with additional enterprise capabilities in enhanced subscriptions. It currently only works with open source technologies like Selenium, Appium and JS Unit Testing. It is the only mobile testing tool that supports automation for native, hybrid and mobile web testing across all device types (real, emulators and simulators).
For more information: Head over to Saucelabs.com to learn more.
Reviews: Visit G2Crowd.com or ITCentralStation.com to read reviews about SauceLabs.
Pricing: Pricing starts at $19 per month with 6 pricing tiers. View pricing here.
Summary: Supports continuous development and testing on real devices. pCloudy also offers location testing, HTML5 browser functionality and debugging capabilities.
Features:
Tool Type: Extension
What you really need to know: Smart Software Testing Solution’s pCloudy emphasizes continuous development and testing as well as debugging.
For more information: Checkout their website here.
Pricing: Pricing starts at $20 per 5 hours with 6 pricing tiers. View pricing here.
Summary: Provides native user interfaces, native API access and native performance for mobile app testing. Xamarin Platform also allows for shared projects and portable class libraries and supports testing on iOS, Android and Mac. Its testing model uses the C# language.
Features:
Tool Type: Extension
What you really need to know: Xamarin allows for native development and testing across iOS, Android and Mac, however it uses the C# language exclusively.
For more information: Visit Microsoft’s Visual Studio site to learn more.
Test data management tools automate the process of deploying “gold” test data to databases. The introduction of open source databases first shook up the market, and it is facing change again due to the rise of NoSQL/Big Data stores. Let’s look at the top Test data management tools to consider.
Summary: Offers sensitive data discovery and classification, data masking, test data generation, data subsetting, data connectivity, pre-built application accelerators and monitoring and compliance reporting. Informatica TDM can be deployed on premise, in the cloud or using a hybrid cloud model.
Features:
What you really need to know: Informatica offers a best of breed solution around data security and data management. Its test data management solution offers persistent data masking, test data generation, data subsetting and cloud and warehouse capabilities.
For more information: Visit the Informatica TDM product page.
Summary: A test-oriented and fully automated solution that is built for reusability. Test Data Manager provides synthetic data generation, data masking, data subsetting, coverage analysis, test data allocation, test data warehouse and virtualization. It is also geared toward the Agile model of continuous delivery.
Features:
What you really need to know: CA Technologies is highly focused on test data management as it relates to DevOps and automation. Its Test Data Manager solution, which came from a 2015 acquisition of GridTools, provides data masking, data provisioning and test data generation.
For more information: Learn more on the Broadcom website.
Summary: Uses pre-built workflows and services on demand for continuous testing and Agile development. IBM InfoSphere Optim makes it easy to create production-like environments, allows for functional, regression integration and load testing via integrations with the Rational Test Workbench and allows for data masking and enterprise test data management policy development and enforcement.
Features:
What you really need to know: IBM gained test data management capabilities in 2005 through its acquisition of Ascential Software. Its InfoSphere Optim test data management solution supports data masking, data subsetting and data generation and integrates with the Rational Test Workbench.
For more information: Visit the IBM InfoSphere page for more information.
Summary: Delphix Engine is a virtualization engine that streamlines data delivery, compresses and creates virtual copies of production data and captures changes at the transaction level. It offers self-service data management and can be used on premise or in the cloud. Delphix Data Masking works alongside the Delphix Engine to securely mask data by replacing sensitive data with fictitious data to better protect data in downstream, non-production environments.
Features:
What you really need to know: Delphix is the newest established test data management player, though it has made great strides since its 2008 founding. The Delphix Engine and Data Masking tools support data masking, data subsetting and data generation and are compatible with AWS.
For more information: Head over to Delphix.com to learn more.
Environment management tools automate the process of deploying software into a test or staging environment. These tools are increasing in importance alongside the rise of DevOps due to quicker builds and the need for automation. Docker, Puppet and Chef have paved the way for newer, cheaper and better environment management solutions.
Here’s a list of top Environment Management tools to consider:
Summary: A SaaS-based tool that allows for management of both standalone and integrated environments throughout pre-production. Plutora TEMS includes an environment availability and allocation engine, a unified dashboard for environment-related information, governance processes to identify and fix changes in test environments and integrations with several leading tools. In addition to environment scheduling and configuration, it also provides cost forecasting.
Features:
What you really need to know: Plutora TEMS is a comprehensive, enterprise-ready testing tool that offers a variety of capabilities along the entire testing lifecycle, including auditing and governance as well as customizable management processes.
For more information: Visit the TEMS website for more information.
Summary: A web-based tool that supports continuous integration and regular testing. Eggplant Manager works with Eggplant Functional and Eggcloud to enable automated testing. It allows for defining of test runs, scheduling test executions, managing systems and devices and analyzing results.
Features:
What you really need to know: TestPlant eggPlant Manager works well for environment management for businesses already using other TestPlant solutions. It can be limiting otherwise.
For more information: Head over to TestPlant.com for more information.
Pricing: No prices listed, but view licensing and pricing information here.
Summary: A SaaS tool that provides a single source of truth for build versions on all environments. Omnium allows for capturing, organizing and identifying usage and scheduling patterns for applications and IT environments. It can be deployed locally or via the cloud and is available in two versions: Omnium Light and Omnium Enterprise.
Features:
What you really need to know: TEMS Omnium is available in two versions that each meet different levels of needs. Its most unique feature is the wide variety of integrations with DevOps and TestOps tools.
For more information: Learn more about Omnium here.
Pricing: Plans start at $55 per mo with 2 pricing tiers. View pricing here.
Mobile testing has become increasingly critical as mobile device usage grows ubiquitous. Given the variety of application types (native, hybrid, mobile web) and operating systems, testing mobile applications can prove difficult. Mobile testing tools use automated testing frameworks to help simplify this process and we’ve outlined the op mobile software testing tools for you below.
Extra: Are you ready for mobile? Take a look at our take on why mobile testing and Sauce Labs are a match made in the cloud[s]
Summary: Offers real devices, including phones and tablets, for mobile testing. DeviceAnywhere is part of Keynote’s larger Mobile Testing tool, which provides test automation and supports unit, build acceptance, regression and explorative and negative testing.
Features:
What you really need to know: Keynote DeviceAnywhere allows for automated testing across a wide variety of devices and supports a multitude of integrations that allow the tool to work seamlessly with an Agile development process.
For more information: Learn more on the DeviceAnywhere product page.
Pricing: Plans start at $200 per project with 5 pricing tiers. View pricing here.
Summary: Allows for mobile testing using both real devices and emulators. For mobile testing, Eggplant applies Eggplant Functional’s image-based UI approach to simplify cross-device and cross-platform mobile testing.
Features:
What you really need to know: TestPlant eggPlant Mobile supports test automation for mobile testing across various devices and platforms, with the option to use real devices or emulators.
For more information: Check out the Eggplant Functional testing tool here.
Summary: An open source test automation framework for mobile testing. Appium uses the WebDriver protocol to support testing for iOS and Android apps and works with native, hybrid and mobile web apps. It also offers full access to back-end APIs and supports a variety of programming languages.
Features:
What you really need to know: Appium is an open source framework that allows for automated testing for native, hybrid or mobile web apps for iOS and Android. It does not require code modifications and supports testing in any language.
For more information: Head over to Appium.io to learn more.
Reviews: Checkout Appium reviews on ITCentralStation.com
Summary: Provides test automation for native, hybrid and mobile web apps for Android devices only. Selendroid is built on Selenium and the WebDriver API. It does not require any code modifications to test apps and offers a built-in inspector to simplify test case development.
Features:
What you really need to know: Selendroid offers automated testing for native, hybrid or mobile web apps for Android only. It supports multiple Android target APIs and can be extended at runtime via extensions.
For more information: Checkout Selendriod.io to learn more.
Reviews: Head over to G2Crowd.com to read reviews.
Summary: Provides test automation for native, hybrid and mobile web apps for iOS devices only. ios-driver uses Selenium and the WebDriver API. It does not require any code modifications to test apps.
Features:
What you really need to know: ios-driver offers automated testing for native, hybrid or mobile web apps for iOS only. It does not require jailbreaking of devices and can be used as part of a Selenium Grid to manage a device or simulator farm.
For more information: Learn more about ios-driver here.
Pricing: ios-driver is a free download.
Bug tracking tools simplify the process of identifying, managing and reporting on software bugs. These tools strengthen collaboration, streamline recording and reporting on bugs and ensure consistency across the board, especially when the alternative is a spreadsheet. Checkout the top bug tracking tools to consider below:
Summary: An open source bug tracking tool that undergoes regular development. Bugzilla offers tracking for bugs and code changes, collaboration for team members, patch submission and reviewing, integrated email capabilities and more.
Features:
What you really need to know: Bugzilla is a free, open source bug tracking tool that offers a variety of capabilities around bug tracking and user collaboration. It is under active development.
For more information: Head over to the Bugzilla website.
Reviews: Head over to G2Crowd.com to read reviews.
Pricing: Bugzilla is a free tool.
Summary: An open source issue tracker that allows for custom issue fields, notifications and workflows. MantisBT also provides email notifications and role based access control to make collaboration easy.
Features:
What you really need to know: MantisBT is a simple yet powerful open source bug tracking tool. It is free for use and offers a variety of plugins to extend its functionality.
For more information: Learn more about MantisBT here.
Reviews: Head over to G2Crowd.com to read reviews.
Pricing: MantisBT is a free tool.
Summary: Provides bug tracking and issue tracking as part of a larger incident management tool. JIRA offers capabilities around planning, tracking, releasing and reporting and supports both custom and out-of-the-box workflows. It also integrates with a variety of testing tools and add-ons, available via the Atlassian Marketplace.
Features:
What you really need to know: JIRA provides bug tracking along with a variety of other incident management capabilities that can extend its use beyond software development. It is an enterprise-ready solution that supports a wide variety of integrations.
For more information: Head over to Atlassian to learn more about JIRA.
Reviews: Read reviews at G2Crowd.com, Softwareadvice.com, and GetApp.com
Pricing: Plans start at $10 per user with 9 pricing tiers. View pricing here.
Summary: Provides bug tracking as part of a larger work tracking and collaboration tool. FogBugz supports task management, time tracking, smart scheduling, email support, document collaboration and crash reporting. It also helps manage Agile development and review and manage code.
Features:
What you really need to know: FogBugz provides bug tracking as well as numerous other work tracking and collaboration capabilities that expand its use cases beyond software development. It is very collaboration-friendly.
For more information: Learn more a the FogBugz website.
Pricing: Plans start at $20 per mo with 2 pricing tiers. View pricing here.
Summary: Offers issue tracking and project management. Bontq can capture screenshots and record videos, supports team collaboration and management and delivers email notifications regarding changes.
Features:
What you really need to know: Bontq is a cloud-based bug tracking and project management tool with a desktop component. It offers recording capabilities as well as task management and documentation.
For more information: Visit Bontq.com to learn more.
Pricing: Plans start at $9 per mo with 5 pricing tiers. Request pricing here.
Summary: Provides customizable bug tracking capabilities, including bug submissions, user discussions, bug tracking and prioritization, dynamic reporting, team management and email alerts. BugAware is available as a web-based or on premise tool.
Features:
What you really need to know: BugAware provides highly customizable bug and issue tracking with advanced search features. Because of its team management, reporting and customization capabilities, its use cases extend beyond software testing.
For more information: Head on over to BugAware.com to learn more.
Pricing: Plans start at $12 per mo with 2 pricing tiers. Request pricing here.
Niche categories are emerging to handle the new problems being created from Agile, mobile, cloud, DevOps and best of breed movements. The top emerging niche fields include:
Test reporting tools centralize and standardize the reporting around testing activity from independent tools, bridging the Agile/Waterfall, cross-platform and manual vs. automated testing metrics divides. With the right test reporting tools teams can ditch spreadsheet and start gaining real-time insight into the progress, health and quality of software testing. Checkout the top test reporting tools to consider below:
Summary: Provides self-service business intelligence for test metrics and reporting. qTests Insights offers a single source of truth for teams to manage and analyze metrics. It includes drill down reporting, custom dashboard capabilities, visual heat maps and interactive charts.
Features:
What you really need to know: Tricentis recently revamped its qTest reporting tool and the result is qTest Insights 2.0. It used this revamp to add unique features like heat maps to better visualize data and to improve overall capabilities to provide more accurate, real-time insights in an easy to use format.
For more information: Head over to the qTest Insights product page.
Summary:A SaaS test management tool with both web and desktop clients that supports various testing methodologies, offers test management capabilities throughout the entire testing lifecycle and supports automated testing. Testuff also offers defect reporting, including two-way integrations with bug trackers and video tracking of defects.
Features:
What you really need to know: Testuff is not solely focused on test reporting. Rather, it is a test management tool that includes defect reporting. It also offers a multitude of integrations with bug trackers and several integrations with automation tools.
For more information: Visit Testuff.com to learn more.
Logging/debugging tools collect information around technical and functional errors from pre-production and production environments. Production logging is mostly useful in mobile and cloud applications. Checkout the logging/debugging tools to consider below.
Summary: Supports exploratory, session-based and manual testing across platforms by automatically recording test sessions and documenting defects. QASymphony qTest eXplorer integrates with leading Agile ALMs, including Rally, JIRA and VersionOne, to improve issue and defect submission.
Features:
What you really need to know: QASymphony qTest eXplorer eliminates the need for manual screen grabs and documentation by automating the entire process. This automation as well as integrations with leading Agile ALMs allows for streamlined processes, deeper investigations for bugs and more robust documentation.
For more information: Checkout the qTest eXplorer product page to learn more.
Summary: Captures user actions during manual exploratory testing and provides write ups to enable accurate bug reproduction. HP Sprinter also provides automated defect scanners for certain conditions like spelling, mirror testing to run tests across multiple machines and data injection to eliminate the need to repeat manual tests.
Features:
What you really need to know: HP Sprinter was released in 2011 as part of HP QC/ALM Premium Edition. In addition to capturing screenshots and videos, HP Sprinter also provides much more functionality around test automation, data injection and more.
For more information: Learn more about Sprinter here.
Summary: Focused only on mobile apps, Instabug provides in-app feedback and bug reporting and enables live chat with beta testers directly inside the app. Instabug also offers crash reporting, workflow automation, dozens of integration opportunities and a single dashboard for bug reporting and fixing.
Features:
What you really need to know: Released in 2013, Instabug focuses solely on mobile applications and offers advanced features like live chat with beta users and crash analytics.
For more information: Head over to Instabug.com to learn more.
Reviews: Head over to G2Crowd.com to read reviews.
Pricing: Plans start at $49 per mo with 4 pricing tiers. View pricing here.
Summary: Provides video recording, logs and user feedback solely for iOS and Android mobile applications. TestFairy also offers app distribution, crash reports and community pages.
Features:
What you really need to know: Released in 2013, TestFairy is comparable to Instabug, as it also focuses solely on mobile. However, TestFairy is focused more on bug reporting than bug management compared to Instabug.
For more information: Checkout TestFairy.com to learn more.
Pricing: Plans start at $79 per mo with 4 pricing tiers. View pricing here.
Summary: Provides live interactive sessions for cross-browser testing with responsive testing and SSH tunnels for local testing. Browslering also offers screenshots, screen sharing, a bug hunter and videos. It is for web testing only.
Features:
What you really need to know: Founded in 2011, Browserling focuses only on web and does not support automated testing.
For more information: Learn more at Broswerling.com
Reviews: Read reviews on the Google Chrome Store.
Pricing: Plans start at $19 per mo with 2 pricing tiers. View pricing here.
Summary: Provides testing for a range of devices and browsers as well as local testing to test development and internal sites. BrowserStack also offers a native experience, with all testing taking place on real devices and browsers, not virtual machines.
Features:
What you really need to know: Founded in 2011, BrowserStack allows for testing on both the web and real mobile devices.
For more information: Learn more at Broswerstack.com.
Reviews: Read reviews over at TrustRadius.com and ITCentralStation.com.
Pricing: Plans start at $29 per year with 3 pricing tiers. View pricing here.
Summary: Provides live testing on real browsers and mobile devices via an interactive test lab. CrossBrowserTesting also offers automated screenshots and screenshot comparisons, Selenium tests and local tests.
Features:
What you really need to know: SmartBear acquired CrossBrowserTesting in 2016 and it is not yet fully integrated into the SmartBear environment. On its own, CrossBrowserTesting allows for testing on both the web and real mobile devices.
For more information: Learn more by visiting the CrossBrowserTesting website.
Pricing: Plans start at $29 per mo with 3 pricing tiers. Visit their pricing page here.
Crowd testing tools use “the crowd” to assist in software testing through web-enabled platforms. Crowd testing reduces the general cost of labor as well as the cost of access to specialized devices, networks and skill sets. Checkout the top Crowd Testing tools to consider below, but first:
Crowd testing communities are either managed or unmanaged. A managed community is one where the vendor manages interactions (e.g. bug verification) with the crowd. These communities minimize the effort required from your business and provide a much higher quality service, but they are also more expensive. Meanwhile, an unmanaged community is one where your business manages interactions with the crowd. These communities require more effort from your business, but they are also less expensive.
Summary: Crowd sources automation, functional, security, load, localization and usability testing. 99tests works with over 20,000 testers across over 150 countries.
Features:
Community Type: Unmanaged
What you really need to know: 99tests offers unmanaged community crowd testing for a wide variety of tests and draws on a global community of testers.
For more information: Visit 99tests.com to learn more about their crowd testing options.
Pricing: Plans start at $3995 per test cycle with 2 pricing tiers. View pricing here.
Summary: Provides crowd testing for exploratory functional testing, test case execution, accessibility and security, usability, localization and load testing. Applause relies on a network of over 250,000 professional testers globally.
Features:
Community Type: Managed and unmanaged, moving toward fully managed
What you really need to know: Applause is the largest player in the crowd testing space, as it is well-established and has an extensive base of testers. It currently offers both managed and unmanaged communities, but it is moving toward more premium managed offerings.
For more information: Learn more on the Applause website.
Reviews: Head over to G2Crowd.com or TrustRadius.com to read reviews.
Pricing: Visit their pricing estimator page.
Summary: Provides a marketplace of testing specialists who pitch and bid on testing projects of all kinds.
Features:
Community Type: Managed
What you really need to know: blur Group uses a bidding model to give projects to the best bidders based on their overall offer, including project pitch and price. Companies have hands-on selection with blur Group managing the process.
For more information: Checkout the blur Groups crowd testing options.
Reviews: Head over to TrustPilot.com to read reviews.
Summary: Delivers crowd testing for application security through a network of over 28,000 security researchers. Bugcrowd uses a bounty model to encourage testers to find more bugs.
Features:
Community Type: Unmanaged
What you really need to know: Bugcrowd specializes in finding a small volume of critical security bugs via a bounty system that rewards more established testers.
For more information: Learn more at Bugcrowd.com.
Summary: Provides functional, usability, security, localization, lifestyle, accessibility and in-live performance testing. Every BugFinders project includes a Test Project Manager who provides regular status updates and other critical information. BugFinders also compiles a report of the testing outcomes.
Features:
Community Type: Managed
What you really need to know: BugFinders focuses on delivering white glove service through its crowd testing, with daily updates and report analysis walkthroughs for every project.
For more information: Visit BugFinders.com to learn more.
Summary: Crash reporting for iOS and Android applications. Crashalytics is lightweight with a negligible network impact that offers advanced reporting with real-time updates.
Features:
What you really need to know: Crashalytics reports crashes in mobile apps in real-time. It was acquired by Twitter in 2013.
For more information: Learn more at Crashalytics.com.
Reviews: Head over to G2Crowd.com or GetApp.com to read reviews.
Pricing: This software is free.
Summary: Provides managed communities that offer functional and performance testing as well as test management and test automation.
Features:
Community Type: Managed
What you really need to know: Crowdar does mostly small scale work, with its end-to-end testing services managed via an assigned employee.
For more information: Visit Crowdar to learn more.
Summary: Functional, localization and usability testing for websites, mobile apps (iOS, Android and Windows), video games and software. CrowdSourced Testing relies on a global network of over 45,000 professional testers with an average of 6+ years of experience.
Features:
Community Type: Managed
What you really need to know: CrowdSourced Testing offers functional, localization and usability testing, with a strong focus on UX and localization testing.
For more information: Visit CrowdSourced Testing website to learn more.
Pricing: Pricing starts at $49 per platform/testers with 2 other pricing tiers. View pricing here.
Summary: Provides performance testing and test automation. CrowdTest offers pay per bug or pay per test models and rates and pays its testers based on their performance.
Features:
Community Type: Unmanaged
What you really need to know: CrowdTest is a South American-focused company that uses a community of testers that it rates and pays based on how well they do.
For more information: Go here to learn more about CrowdTest.
Summary: Provides app, localization, non-functional, security, cloud, usability and game testing as well as “bug bashing.” CrowdTesters uses a proposal system where testers submit project proposals based on specified requirements and organizations select their preferred tester.
Features:
Community Type: Unmanaged
What you really need to know: CrowdTesters provides an unmanaged community in which you can submit project requirements and select your tester based on proposals.
For more information: Visit CrowdTesters.com to learn more.
Summary: Provides a large network of freelancers for any number of testing requirements. Upwork is not a testing-specific community, but it does boast a large network of technology freelancers. With Upwork, companies post a job description, freelancers apply and companies can select a freelancer based on factors like skills, project proposal and pricing.
Features:
Community Type: Unmanaged
What you really need to know: Formerly Elance-Odesk, Upwork has a network of over three million freelancers, however only a small portion specialize in software testing. That said, Upwork still has among the largest networks of technology freelancers.
For more information: Checkout the developers section of Upwork.com to find testers.
Summary: Provides distribution, crash reports, user feedback, user metrics and team management. HockeyApp also integrates with several development platforms. Companies simply upload their apps and users can download for testing.
Features:
Community Type: Managed
What you really need to know: Acquired by Microsoft in 2014, HockeyApp provides a very simple and straightforward user experience, although its testing capabilities are limited compared to other tools.
For more information: Head on over to HockeyApp.com to learn more.
Pricing: Plans start at $30 per mo with 5 pricing tiers. View pricing here.
Summary: Offers managed QA services, traditional QA services, mobile app testing and game testing. LionBridge uses both lab-based automated testing and hands-on location testing.
Features:
Community Type: Managed
What you really need to know: LionBridge offers a network of over 140,000 crowd sourcers, however only a portion of these people are testers.
For more information: See their network of crowdsources here.
Summary: Provides functional and usability testing for mobile. Mob4Hire has a network of 65,000 testers, all of whom it ranks in order to eliminate weaker testers.
Features:
Community Type: Unmanaged
What you really need to know: Mob4Hire is a moderately well established testing community that specializes in functional and usability testing for mobile software only.
For more information: Surf the mob of testers at Mob4Hire.com
Summary: Delivers functional, exploratory, website, mobile app and email template crowd testing. MyCrowd boasts a network of 25,000 testers across 40 countries.
Features:
Community Type: Unmanaged
What you really need to know: MyCrowd provides a relatively small community of testers, with a big focus on website and mobile testing.
For more information: Visit mycrowd.com to learn more.
Pricing: Plans start at $23 per hour with 3 pricing tiers. View pricing here.
Summary: Crowd sourced bug finding with support for a variety of mobile networks and hardware. Pay4Bugs integrates with bug tracking systems and follows a pay per bug model.
Features:
Community Type: Unmanaged
What you really need to know: Pay4Bugs offers crowd sourced bug finding for mobile apps on a pay per bug model.
For more information: Visit Pay4Bugs.com to learn more.
Pricing: Plans start at $300 per mo with 3 pricing tiers. View pricing here.
Summary: Provides compatibility, functional, localization, load and performance, usability and security crowd testing. passbrains offers access to a private company-created crowd or a passbrains crowd, with testers selected based on numerous factors like skills and location.
Features:
Community Type: Managed
What you really need to know: passbrains provides a mix of managed onsite, offshore and crowd testing for a wide variety of needs. It is enterprise-focused and built for expertise in mobile crowdsourcing.
For more information: Head over to passbrains.com to learn more about their crowd testing.
Pricing: Plans start at € 990 per with 3 pricing tiers. View pricing here.
Summary: Provides a network of freelancers, including but not limited to software testers. PeoplePerHour allows companies to post a job and review applicants or search for freelancers directly based on skills, rates, etc.
Features:
Community Type: Unmanaged
What you really need to know: PeoplePerHour is a middle-man for finding freelancers. It vets freelancers to ensure it provides quality testers, but projects are not managed.
For more information: Visit peopleperhour.com to learn more.
Summary: Managed crowdsourced testing for cloud, web, mobile and games across functional, usability, compatibility and security testing. Revolution IT is an Australian company with 100,000 testers globally. It offers integrations with testing tools like JIRA and Quality Center as well as a joint offering with HP AppPulse.
Features:
Community Type: Managed
What you really need to know: Revolution IT is an Australian company with a large presence in the local market. It provides a variety of crowd testing capabilities and integration opportunities.
For more information: Learn more here.
Summary: Crowd testing for mobile, websites, games and Mac and PC software. TestBats boasts a network of over 45,000 testers in 178 countries.
Features:
Community Type: Managed
What you really need to know: TestBats is a German crowd testing company and its work is mostly limited to the European market.
For more information: Learn more at TestBats.nl.
Summary: Provides crowd testing solely for iOS and Android mobile applications via video recording, logs and user feedback. TestFairy also offers app distribution, crash reports and community pages.
Features:
Community Type: Managed
What you really need to know: TestFairy provides a detailed video capture of all testing, including statistics like CPU, memory and network.
For more information: Head over to TestFairy.com to learn more.
Pricing: Plans start at $79 per mo with 4 pricing tiers. View pricing here.
Summary: Offers recording for iOS and Android mobile app testing so that customers can easily communicate issues. Testfire also provides video logs and device details as well as integrations with issue trackers like JIRA and GitHub and customer support tools like Zendesk and Desk.com.
Features:
Community Type: Unmanaged
What you really need to know: Testfire is a crowd testing app that specializes in testing documentation via video recording.
For more information: Visit Testfire.io to learn more.
Pricing: Plans start at $495 per mo with 2 pricing tiers. View pricing here.
Summary: Provides crowdsourcing for design and prototyping, application development, algorithms and analytics and app modernization. Topcoder also offers staff augmentation and boasts a community of one million members.
Features:
Community Type: Managed
What you really need to know: Topcoder offers a large, well established network, although testing is not its only focus. It follows a pay per bug model.
For more information: Head on over to Topcoder.com to learn more.
Beta management tools control the beta testing process from app deployment to monitoring by collecting feedback and instructing beta participants how and where to test the application. Let’s take a look at the top Beta Management testing tools to consider:
Summary: Offers a community of users to test and provide feedback on websites, mobile applications and desktop applications. With BetaEasy, users solve problems collectively by communicating with one another and voting on each other’s suggestions. It also allows companies to communicate with users and react to their suggestions and provides detailed reports of all communications and progress.
Features:
What you really need to know: BetaEasy allows for beta testing across applications and websites of all kinds and makes providing feedback a community effort, where testers can bounce ideas off one another and vote on suggestions.
For more information: Visit BetaEasy to learn more.
Pricing: Plans start at $19r mo with 3 pricing tiers. View pricing here.
Summary: Simplifies inviting beta users, distributing builds and collecting feedback for beta testing of mobile apps. Applause Mobile Beta Management is mobile-only. It allows users to share feedback and submit bugs directly from within the app they are testing and provides managers with bug and feedback reports as well as participant session information and automatic crash reporting.
Features:
What you really need to know: Applause Mobile Beta Management supports beta testing for mobile applications only. It offers a branded, enterprise-ready solution that integrates with Applause’s testing suite and leading bug tracking systems to simplify the process of fixing bugs reported by users.
For more information: Visit Applause Mobile Beta Management to learn more.
These companies aren’t quite ready for our ultimate list just yet, but they are definitely ones to keep an eye on.
Learn how AI can be used to eliminate common testing headaches, as well as what the future of autonomous testing holds.
Migrating legacy applications to the cloud is risky, yet those who get there first will reap the greatest rewards. Learn how early adopters use automated testing to accelerate timelines and minimize risk.
Learn how NeoLoad combines performance testing with application performance monitoring insights to take performance engineering to the next level.