Language

En

De

  1. HomeArrow Icon
  2. Case StudiesArrow Icon
  3. Dolby amplifies testing success with ServiceNow automationArrow Icon

Dolby

Dolby amplifies testing success with ServiceNow automation

+

test cases automated for ServiceNow

default

%

tests run offline in unattended mode nightly

default

Company overview

For more than 50 years, Dolby has developed groundbreaking technologies that have revolutionized entertainment and communication experiences for billions of people worldwide in cinema, home theaters, PCs, mobile phones, and games. For more than six years, Dolby’s Testing Center of Excellence (TCoE) has employed an innovative approach to testing with Tricentis Tosca and Tricentis qTest to deliver fast, high-quality releases of several financially significant applications. Applications such as ServiceNow, which are part of critical IT infrastructure, have traditionally fallen outside the scope of the TCoE’s testing practices. But when the infrastructure team needed to speed up ServiceNow testing, they turned to Dolby’s TCoE lead and IT director Mamatha Mitr to investigate Tosca’s capabilities.

ServiceNow is a critical piece of enterprise software that streamlines the organization’s digital workflows and operations to improve efficiency. Its core functionalities extend to many business functions, serving as a single point of contact for service delivery to minimize wait times and interruptions.

  • Arrow Icon
    Industry: Consumer Products
  • Arrow Icon
    Company size: 1,001-5,000 employees
  • Arrow Icon
    Location: United States
  • Arrow IconProducts:

Challenges

  • Each ServiceNow upgrade required nine weeks of manual regression testing by 14 team members
  • Time-consuming manual testing was not an effective utilization of high value resources’ time
  • Manual testing did not provide the complete coverage required for the release
  • Complex processes within ServiceNow are constantly changing, requiring test cases to be frequently updated
  • Testing for different data sets covering all regions was difficult to achieve

ServiceNow’s twice yearly upgrades and quarterly patches required long and rigorous testing to ensure ongoing functionality of complex, constantly evolving processes was complete. These frequent changes meant testing was complicated and required ongoing updates by a team of business analysts, application developers and maintenance specialists. These teams spent over nine weeks completing manual regression testing for each upgrade — a process that proved to be an ineffective use of their time. With ServiceNow’s frequent release schedule, a 9-week testing cycle made it difficult to achieve complete test coverage across the many permutations and combinations of test data and different scenarios across multiple applications and region-specific versions.

Scaling test automation quickly for 90% coverage

The infrastructure team worked with the existing TCoE at Dolby to move forward with Tosca’s scriptless test automation, leveraging documented best practices. They were able to engage a handful of senior consultants from Tricentis in conjunction with a project manager to quickly and significantly scale test automation for ServiceNow through the course of multiple planned sprints. The scope would cover ServiceNow, as well as its integrations with Peoplecare HR and AskIT – Dolby’s internal knowledge and ticketing portal.

The focus of the initial project was to automate as much as possible while ensuring test cases developed were reusable for future upgrades and releases. A risk-based test approach was used to assign priority for which tests would be automated in each sprint. Priority was based on frequency and “damage class” identified by the team. The result was 90%+ automated test coverage for all key areas of the application.

A secondary focus was creating the ability to execute test cases in an offline, unattended mode. This allows test execution to happen during non-working hours and frees up business analysts and application developers to focus on higher priority tasks during the day.

To date, the team has automated 200+ test cases to support future releases of ServiceNow. 95% of those tests are successfully executed in unattended mode on the Tricentis Distributed Execution Server (DEX).

“We defined success metrics before we started the project. If automation can bring down regression testing time to a few hours and minimize defect leakage to production, then the project would be considered successful.”

Extending test coverage with Tricentis Test Data Management

To deliver releases with confidence, different scenarios for the same functionality with data from all major geographies would need to be tested. Automating these scenarios allows testing for various combinations of data on a short timeline. This contributes to increased coverage with different data combinations for each monthly release, which are managed with Tricentis Test Data Management (TDM).

Throughout the project, the team made sure to follow the TDM approach. Mamatha explains, “We wanted an effective way to manage test data and ensure that there are specific steps outlined to recover data and make it available for testing.” Test data includes prerequisite steps for creating and maintaining them for consistent test results.

“After having gone through ServiceNow automation and utilizing this process for a few months, I can tell that we are not spending several cycles on testing, and we have employed a no-touch to a very low-touch model. The team is benefiting from seamless upgrades. We utilize the centralized dashboard that provides complete visibility of the metrics, like total runtime of what has been completed, what’s scheduled to be done, etc.”

Results

  • All key areas of ServiceNow are automated and have 90%+ test coverage
  • Automation was achieved using a slim resource model
  • Risk-based testing approach assigns priority to test creation, ensuring the most business-critical functions are tested first
  • 95% of testing is executed offline in an unattended mode during non-working hours – allowing the team to focus on high priority tasks during the workday
  • Test automation helps streamline ongoing maintenance of ServiceNow
  • Highly reusable test cases are available for all future upgrades and releases
  • 7 different ServiceNow applications and portals automated
  • Test Data Management ensures test data is available for a wide range of test scenarios, extending test coverage
  • Test cases are stored using Tricentis qTest for easy organization, management, sharing, and tracking

Hear from Dolby in their own words

Resources

Recommended for you