Multinational chemical company
How a multinational chemical company achieved 16X more testing at 32% lower costs
x more tests executed
default
faster release cycles
default
lower costs
default
Company overview
These are the staggering results that this $28 billion chemical company specializing in both industrial gas and engineering has achieved over a multi-year testing transformation architected by their forward-thinking quality leaders and powered by Tricentis test automation.
The organization rapidly implemented and scaled automated end-to-end testing across highly customized SAP instances, Salesforce, web apps, mobile apps, and multiple custom/legacy apps. Their strategic, highly efficient testing approach enables the company to deliver internal IT projects on time and on budget—ensuring that continued innovation never places the company’s complex global operations at risk.
- Industry: Energy
- Location: Global
- Products:
Enabling safe innovation across an extremely complex application landscape
The company serves a variety of end markets including chemicals & refining, food & beverage, electronics, healthcare, manufacturing, and primary metals. Their industrial gases are used in countless applications, from life-saving oxygen for hospitals to high-purity & specialty gases for electronics manufacturing, hydrogen for clean fuels, and much more.
They maintain a deep commitment to being the “best performing global industrial gases and engineering company, delivering innovative and sustainable solutions for customers in a connected world.” Achieving this goal requires flawless IT operations. Core business processes must remain reliable and resilient as their extremely complex application stack continuously evolves. Given the scope and scale of their global business, even a subtle application “glitch” could quickly translate to supply chain disruption, regulatory issues, and lost business opportunities.
When two new quality leaders were given the responsibility of the Testing Center of Excellence (TCoE), they faced quite a challenge. The team is responsible for ensuring that custom development and vendor updates do not disrupt critical business processes across 13 SAP instances and key enterprise applications including Salesforce, mobile apps, web apps, and many Microsoft-based applications. The majority of this application portfolio is evolving quite rapidly. SAP applications are typically updated weekly, and many other applications are released in a Continuous Delivery model. However, any change is a risky endeavor. For example, just within the Salesforce platform as a service, the development teams might be using similar objects across 10 applications that are already in production and 10 others that are still under development. All the applications are highly interconnected, and they share the same codebase as well as the same objects. Daily testing is critical for ensuring that one team’s improvements don’t unintentionally disrupt the other teams’ core processes.
Testing transformation was imperative
Like many enterprise companies, the testing process was built to support the first generation of test automation tools: script-based test automation that requires specialized technical expertise and is implemented in months-long waterfall development processes. Tests were created by the TCoE in India with a small team of company employees supplemented by global system integrator (GSI) contractors, who were ramped up and down according to fluctuating project needs.
Although end-to-end business processes span multiple technologies, the team was using different (disconnected) tools to test each technology type. SAP tests used a common script-based commercial testing tool. Salesforce tests were built on Selenium’s open source test automation. Android and iOS mobile apps used a commercial mobile testing tool. Custom legacy applications were often beyond the scope of any of these tools. And yet another commercial tool was used to coordinate all these tests and correlate their diverse results.
The new team leaders—Head of Service Delivery, Application Testing and Security Management – Enterprise Applications) and Head of IT Testing Center of Excellence—soon identified a number of challenges with this approach:
- It was expensive and slow to have the required specialists build the expected level of test automation; a single test suite for a single technology required months to create.
- The brittle scripts required burdensome maintenance to accommodate the changes in each release—so much that it was often simpler to re-create them than to fix them .
- The test specialists could not readily shift from one technology to another (e.g., between SAP, Salesforce, web, and mobile) to suit dynamic project needs or end-to-end testing.
- Their legacy test automation tools were not suited for their recent shift to highly iterative Agile processes and modern DevOps toolchains.
A new test automation tool…
Faced with pressure to ensure the team was testing more, faster, and at a lower cost, the new leaders knew it was time to change the organization’s approach to testing, including its testing products. They started searching for a new testing platform with the following three requirements top of mind:
- Costs: It should enable test automation to be created more efficiently and avoid costly, resource-intensive test maintenance.
- Users: It should support everyone from development teams to functional SME experts and business analysts.
- Technologies: It should test business processes end-to-end, across all the various technologies involved in a single transaction.
An extensive technical analysis revealed that Tricentis Tosca met all of these needs and the team selected it as the tool of choice. They then charted out an ambitious adoption plan. To help accelerate time to value, Tricentis led several pilots to roll out test automation in two phases. The first phase covered test automation for highly customized SAP SRM, SAP ECC, Salesforce, web applications, and pure legacy apps. Phase two involved test automation for mobile and ecommerce apps, script migration, integration with their other tools (Quality Center, Jira, Mobile Labs, …), and exploratory testing.
This was all completed from January to October 2018, with several Tricentis consultants working closely with their internal team—particularly the Head of IT Testing Center of Excellence, who was instrumental to the initiative’s implementation as well as its long-term success. As the Tricentis consultants built a core set of tests for each application; the internal team worked by their side so they knew how to extend and expand this initial foundation.
…And also a new approach: risk-based testing
Leaders knew that their testing transformation required more than a new tool. It also required a more deep-seated process change. Recognizing how risk-based testing could help them maximize test impact while minimizing test creation and maintenance, they worked to change the mindset of the testing organization.
“We have a lot of examples in the past where people were proud of having 3,000+ test cases that ran continuously without uncovering any defects,” one leader explained. “I said, ‘Is that successful? Does that make sense? Don’t you think you have burned resources?’ At the end of the day, you have to search for high-value test automation, and you have to focus on the business risks.”
Of course, for this risk-based testing approach to work, you need someone in the business to help weigh the various requirements’ risks versus one another. The testing team can then prioritize testing for the highest risk functionalities—using formal test case design methodologies to achieve the most effective tests with the fewest test cases. Working in that way, it’s entirely feasible to cover 80% of the business risk with just 20% of the testing effort. This tied in perfectly with the organization’s need to cover more risk, faster, and at a lower cost.
- “One of the key challenges is achieving the right cost balance in test automation. To do this, I need a tool that helps us build test automation quickly, and also helps us maintain that test automation rapidly, efficiently, and sustainably.” ~ Head of Service and Security Management
Test automation grows—and downtime diminishes
The team has steadily expanded the scope of the test automation transformation to cover additional applications as well as the continuous changes being introduced into the applications with pre-existing test automation. Development teams appreciate how the fast feedback on their updates helps them fix issues when it’s easiest and cheapest to do so.
Ultimately, extending test automation and ensuring it remains in sync with application changes has resulted in near-zero downtime—reducing distractions for developers and IT operations while ensuring reliable operations for the broader company and its business partners.
Taking efficiency to new levels
On the heels of a $73B merger, the newly integrated company started focusing on maximizing synergies and efficiencies in 2019. By building on this effective test automation foundation, existing leaders have been able to drive an extremely lean testing practice for the business now that test automation is much simpler to create and maintain, labor costs are significantly reduced. In fact, their labor costs are now just a small fraction of the average at comparable organizations. On top of that, cost avoidance (the costs avoided by their process catching defects prior to production) adds annual savings estimated to be in the millions, according to industry benchmarks. Moreover, testers can easily shift focus and technologies to accommodate variable project lifecycles. This flexibility is key for ensuring that every IT project has the resources required to assess its business risks before it’s released into production.