Case Study

Global Bank

Automating Testing for Windows 10 Upgrades
image

Accelerating and Scaling Testing for Window Upgrades

Background

The team at one of the world’s largest banks was receiving thousands of requests per year to perform validation testing on Windows updates.  Each time a new Windows patch was released, they were expected to check whether critical operations continued to work as expected, plus provide detailed documentation of any issues they encountered. At first, this was feasible. However, as more and more departments and domains required testing, their legacy testing approach could not keep up.

To this point, they had been using HP UFT with limited success for testing the Windows upgrades. Since the existing scripts were difficult to create, maintain, and extend, they decided to search for a scriptless test automation approach that was better suited to their team’s expertise and needs.

Challenges
  • The amount of testing doubled, and the frequency of testing increased by orders of magnitude
  • Existing HP UFT scripts were too brittle for reuse or extension
  • The coverage from existing HP UFT scripts was diminishing due to an increasing number of false positives and the lack of specialized resources required to update the scripts
  • Building new HP UFT scripts required scripting expertise, which was not a common skillset in their team
  • Manual testing was attempted, but only a fraction of the expected tests could be completed in the available timeframe
Scriptless Test Automation with Tricentis

After evaluating several test automation and RPA solutions, the organization selected Tricentis. Without scripting, the team members could rapidly define resilient automated tests that expose business-critical changes—but know to ignore acceptable UI variations.  The end-to-end automation covers everything from test environment setup (including pushing the updates), to executing operations with realistic test data, to validating that critical checkpoints (UI as well as back-end) meet expectations. They were ultimately able to achieve greater business risk coverage with 67% fewer test cases.

Results
  • Team members can rapidly create and manage sophisticated automated tests—without waiting on “technical” resources
  • If the application under test changes, they simply update the model once; all impacted test cases are automatically synchronized
  • Tricentis test data management provides on-demand access to the test data required to complete tests, eliminating month-long provisioning delays
  • Risk-based testing provides immediate insight into whether a given test failure is truly a show-stopper
  • When manual testing is required, documentation is recorded automatically