ServiceNow’s twice yearly upgrades and quarterly patches required long and rigorous testing to ensure ongoing functionality of complex, constantly evolving processes was complete. These frequent changes meant testing was complicated and required ongoing updates by a team of business analysts, application developers and maintenance specialists. These teams spent over nine weeks completing manual regression testing for each upgrade — a process that proved to be an ineffective use of their time. With ServiceNow’s frequent release schedule, a 9-week testing cycle made it difficult to achieve complete test coverage across the many permutations and combinations of test data and different scenarios across multiple applications and region-specific versions.
Scaling test automation quickly for 90% coverage
The infrastructure team worked with the existing TCoE at Dolby to move forward with Tosca’s scriptless test automation, leveraging documented best practices. They were able to engage a handful of senior consultants from Tricentis in conjunction with a project manager to quickly and significantly scale test automation for ServiceNow through the course of multiple planned sprints. The scope would cover ServiceNow, as well as its integrations with Peoplecare HR and AskIT – Dolby’s internal knowledge and ticketing portal.
The focus of the initial project was to automate as much as possible while ensuring test cases developed were reusable for future upgrades and releases. A risk-based test approach was used to assign priority for which tests would be automated in each sprint. Priority was based on frequency and “damage class” identified by the team. The result was 90%+ automated test coverage for all key areas of the application.
A secondary focus was creating the ability to execute test cases in an offline, unattended mode. This allows test execution to happen during non-working hours and frees up business analysts and application developers to focus on higher priority tasks during the day.
To date, the team has automated 200+ test cases to support future releases of ServiceNow. 95% of those tests are successfully executed in unattended mode on the Tricentis Distributed Execution Server (DEX).
- All key areas of ServiceNow are automated and have 90%+ test coverage
- Automation was achieved using a slim resource model
- Risk-based testing approach assigns priority to test creation, ensuring the most business-critical functions are tested first
- 95% of testing is executed offline in an unattended mode during non-working hours – allowing the team to focus on high priority tasks during the workday
- Test automation helps streamline ongoing maintenance of ServiceNow
- Highly reusable test cases are available for all future upgrades and releases
- 7 different ServiceNow applications and portals automated
- Test Data Management ensures test data is available for a wide range of test scenarios, extending test coverage
- Test cases are stored using Tricentis qTest for easy organization, management, sharing, and tracking
Extending test coverage with Tricentis Test Data Management
To deliver releases with confidence, different scenarios for the same functionality with data from all major geographies would need to be tested. Automating these scenarios allows testing for various combinations of data on a short timeline. This contributes to increased coverage with different data combinations for each monthly release, which are managed with Tricentis Test Data Management (TDM).
Throughout the project, the team made sure to follow the TDM approach. Mamatha explains, “We wanted an effective way to manage test data and ensure that there are specific steps outlined to recover data and make it available for testing.” Test data includes prerequisite steps for creating and maintaining them for consistent test results.
“After having gone through ServiceNow automation and utilizing this process for a few months, I can tell that we are not spending several cycles on testing, and we have employed a no-touch to a very low-touch model. The team is benefiting from seamless upgrades. We utilize the centralized dashboard that provides complete visibility of the metrics, like total runtime of what has been completed, what’s scheduled to be done, etc.”