1. HomeArrow Icon
  2. Case StudiesArrow Icon
  3. Bank of Queensland delivers Quality as a Service with continuous test automation and management—in the cloud and beyondArrow Icon

Bank of Queensland

Bank of Queensland delivers Quality as a Service with continuous test automation and management—in the cloud and beyond

%

reduction in regression testing time

default

%

increase in release frequency

default

Company overview

Founded in 1874, Bank of Queensland (BoQ) is one of Australia’s leading regional banks, and they take pride in offering consumers an alternative to the country’s four mega banks. Over the years, BoQ has acquired numerous niche financial providers and retail banking institutions and continues to operate many of them as standalone brands.

BoQ’s most recent acquisition, Members Equity (ME) Bank, firmly positioned BoQ as an innovative bank of choice for younger, tech-savvy customers, with this move to the digital retail space. The bid aligns with BoQ’s forward-thinking philosophy and fits into a broader years-long digital transformation effort. BoQ is the only bank with plans to move the core systems for all four of its brands into the cloud—a huge migration of assets and diverse, complex systems.

“We took advantage of everyone else’s digital transformation model, where we effectively don’t host anything internally and we rely on our partners to do a lot of the hosting and management of our non-production environments,” says Jay Williams, BoQ’s Test Automation Lead. “Now that we’ve proven we can build new banks in the cloud, deliver and operate them through the cloud, the next step is tying the whole solution together.”

Since coming on board with BoQ in 2019, Williams has championed the Agile way and positioned QA as a driver of quality in BoQ’s transformation. He brought change and release teams into the testing conversation and built testing outcomes into BoQ’s Ways of Working and Definitions of Done. He worked to understand the demands of compliance and baked them into the technical requirements of each build.

“In the past, quality was a lever to be pulled, on the same level as time, cost, and scope.” Williams said, “but it doesn’t matter if you’re on time, under budget, and at scope if the build doesn’t work. If a mobile app doesn’t work, people won’t try it again. Quality must be there.”

To execute BoQ’s migration and manage the vast array of toolsets and frameworks used by its many brands, Jay Williams sought to centralize test management and standardize processes to enable continuous testing and faster feedback loops between dev and testing teams.

Creating enterprise-wide alignment is hard enough for one brand, let alone four. Jay Williams and the BoQ team had their work cut out for them.

Digital Transformation through centralized, continuous End-to-End testing

Bank of Queensland wanted to take advantage of third-party SaaS and PaaS platforms where they could use single platforms across multiple applications to establish some commonality and reduce technical complexity.

Utilizing Tosca and qTest, Bank of Queensland delivers Quality as a Service by centralizing its testing model across all platforms and looking at quality and testing at a portfolio level rather than the individual programs, change, or streams of work levels. This centralization also lets Jay Williams and his team standardize automation metrics.

Bank of Queensland was able to implement end-to-end continuous automation, run daily health checks across the majority of non-production environments, and schedule automation runs each morning to uplift their CI/CD processes. This allowed them to provide quality feedback to their developers promptly and shift their load testing further left.

Jay Williams highlighted that the ability to centralize their testing not only reduced test cycle time, but also helped identify the most ideal window. While most organizations run tests at night, when systems are far less occupied, banking systems keep working to process batch runs and payments. Run smoke tests too early, and issues would arise between test completion and the next morning.

“We started kicking them off at night, but after we finished, third-party servers would go down or restart improperly. Every morning there was a scramble to figure out what’s wrong,” he said. “Now we run tests in the morning, from 3 a.m. to 9:30. We can pinpoint issues and assign people to fix them, rather than find them.”

Challenges

  • Legacy, monolithic banking systems limited the testing speed and resulted in a slow release cycle to one big quarterly release.
  • Requirements traceability managed with spreadsheets made QA-dev communication a chore.
  • Dependence on dedicated networks meant that some apps could only be tested in certain locations, adding to test complexity.
  • No way to validate or verify testing environments before the start of testing.
  • Limited time to mitigate issues when load testing is left until the end of the bank’s legacy waterfall process.
  • Lack of testing centralization made applying changes across similar environments challenging, especially with limited test and non-production environments available.
  • A vast and diverse toolkit across several bank brands made testing challenging to manage and clouded visibility into release readiness.

Improved quality and speed of execution

Tosca’s low-code and codeless functionalities empowered Bank of Queensland’s manual testers with limited automation experience to build out their automation suites without the support of more experienced developers.

“Tosca’s UI-level automation is key.” Williams shared, “It was way easier to get my team onboarded with Tosca than it would have been with Selenium. Even testers with no automation experience could easily build their own automation suites.”

Williams also notes that one of the most significant benefits he noticed with implementing qTest was the ability to move requirement tracing and management from Excel spreadsheets into Jira. This reduced double-handling allows linking between test cases and requirements, and gives their testers visibility to see which test cases and executions are being run across different business requirements.

With these process improvements, Bank of Queensland have seen increased ROI through test automation by reducing the time it takes to run regression and ultimately gaining confidence in a new build. Before implementing Tosca and qTest, it would take Bank of Queensland 5-6 weeks from the build stage until they were confident that the build was ready for release.

Within 18 months of partnering with Tricentis, BoQ has reduced its full regression cycle down to under five days—an 80% reduction—giving them a clear view of their new build and providing confidence in those build’s releases.

Results

  • Reduced regression test cycle from 5+ weeks to 5 days
  • Increased release frequency from every 3-4 months to monthly
  • Scaled test automation with existing team members using Tosca’s codeless solution
  • Moved to daily health checks across non-production environments
  • Implemented the scaled agile framework (SAFe) despite operating in a heavily regulated industry
  • Streamlined dev-QA collaboration with test management that integrates across Jira, Tosca, and other test automation tools
  • Improved traceability and release confidence on a much shorter timeline
  • Centralized quality and standardized automation metrics

Looking to the future

With Tosca and qTest helping the team centralize testing and prioritize quality, Williams sees his team implementing more Tosca features to improve their testing further. This includes using service virtualization to simulate environments, systems, and third-party apps, and Tosca’s Vision AI to increase test automation.