How Telia is modernizing performance testing with Tricentis NeoLoad
Hear from Telia, one of Sweden’s largest telecommunications companies, how they use Neoload for their performance testing.
test cases automated for ServiceNow
tests run offline in unattended mode nightly
For more than 50 years, Dolby has developed groundbreaking technologies that have revolutionized entertainment and communication experiences for billions of people worldwide in cinema, home theaters, PCs, mobile phones, and games. For more than six years, Dolby’s Testing Center of Excellence (TCoE) has employed an innovative approach to testing with Tricentis Tosca and Tricentis qTest to deliver fast, high-quality releases of several financially significant applications. Applications such as ServiceNow, which are part of critical IT infrastructure, have traditionally fallen outside the scope of the TCoE’s testing practices. But when the infrastructure team needed to speed up ServiceNow testing, they turned to Dolby’s TCoE lead and IT director Mamatha Mitr to investigate Tosca’s capabilities.
ServiceNow is a critical piece of enterprise software that streamlines the organization’s digital workflows and operations to improve efficiency. Its core functionalities extend to many business functions, serving as a single point of contact for service delivery to minimize wait times and interruptions.
ServiceNow’s twice yearly upgrades and quarterly patches required long and rigorous testing to ensure ongoing functionality of complex, constantly evolving processes was complete. These frequent changes meant testing was complicated and required ongoing updates by a team of business analysts, application developers and maintenance specialists. These teams spent over nine weeks completing manual regression testing for each upgrade — a process that proved to be an ineffective use of their time. With ServiceNow’s frequent release schedule, a 9-week testing cycle made it difficult to achieve complete test coverage across the many permutations and combinations of test data and different scenarios across multiple applications and region-specific versions.
The infrastructure team worked with the existing TCoE at Dolby to move forward with Tosca’s scriptless test automation, leveraging documented best practices. They were able to engage a handful of senior consultants from Tricentis in conjunction with a project manager to quickly and significantly scale test automation for ServiceNow through the course of multiple planned sprints. The scope would cover ServiceNow, as well as its integrations with Peoplecare HR and AskIT – Dolby’s internal knowledge and ticketing portal.
The focus of the initial project was to automate as much as possible while ensuring test cases developed were reusable for future upgrades and releases. A risk-based test approach was used to assign priority for which tests would be automated in each sprint. Priority was based on frequency and “damage class” identified by the team. The result was 90%+ automated test coverage for all key areas of the application.
A secondary focus was creating the ability to execute test cases in an offline, unattended mode. This allows test execution to happen during non-working hours and frees up business analysts and application developers to focus on higher priority tasks during the day.
To date, the team has automated 200+ test cases to support future releases of ServiceNow. 95% of those tests are successfully executed in unattended mode on the Tricentis Distributed Execution Server (DEX).
“We defined success metrics before we started the project. If automation can bring down regression testing time to a few hours and minimize defect leakage to production, then the project would be considered successful.”
To deliver releases with confidence, different scenarios for the same functionality with data from all major geographies would need to be tested. Automating these scenarios allows testing for various combinations of data on a short timeline. This contributes to increased coverage with different data combinations for each monthly release, which are managed with Tricentis Test Data Management (TDM).
Throughout the project, the team made sure to follow the TDM approach. Mamatha explains, “We wanted an effective way to manage test data and ensure that there are specific steps outlined to recover data and make it available for testing.” Test data includes prerequisite steps for creating and maintaining them for consistent test results.
“After having gone through ServiceNow automation and utilizing this process for a few months, I can tell that we are not spending several cycles on testing, and we have employed a no-touch to a very low-touch model. The team is benefiting from seamless upgrades. We utilize the centralized dashboard that provides complete visibility of the metrics, like total runtime of what has been completed, what’s scheduled to be done, etc.”