decrease in testing time
(from 1,300 hours manually to 32 with test automation)
USD saved in testing costs
Duke Energy is one of the largest energy holding companies in the United States – providing electricity to 7.8 million customers and supplying an additional 1.6 million customers with natural gas. Duke Energy is focused on “transforming the customers’ experience, modernizing their energy grid, and generating cleaner energy to create a smarter energy future for their customers and communities.”
Customer engagement is vital to Duke, and it operates customer service centers in each state it serves. An important first step in the mission to transform the customer experience started with streamlining internal billing systems for a universally optimized customer experience.
Duke covers multiple territories and states – Indiana, Ohio, Kentucky, Tennessee, North Carolina, South Carolina, and Florida – all of which came together through a series of mergers. With these mergers, Duke was managing four different billing systems to service 7+ million customers. Existing systems were all built on legacy mainframes and technology that was 20+ years old – some of which still included greenscreens. Duke knew it needed to consolidate this complex application stack into a single billing system to create a common billing experience for every customer reaching a call center.
“We needed to service all of our customers across our states in any jurisdiction and know our customers, no matter where they are,” said Debbie Smith, Senior IT Manager of the SAP Customer Connect System, Duke Energy. “If they’re in North Carolina and Florida, we know them as one customer. Whereas in the past, they were two customers to us, and we didn’t realize who they were as a single entity.”
Duke underwent an evaluation to determine the right platform to support their reimagined business processes — for billing and all internal processes. They ultimately decided to move to a greenfield installation of SAP S/4HANA, including SAP for Utilities, SAP Marketing, and SAP Commerce Cloud. With this new digital core, Duke could consolidate the four legacy billing systems into a single engagement platform while also focusing on data and analytics, customer engagement, and meter-to-cash processes.
“We brought in SAP to help us to transform four mainframe billing systems that had been in existence for over 20 years into a single SAP solution for all of our jurisdictions, which was a huge step in the transformation of our customer efforts that we had on our roadmap,” said Smith.
Over a journey of five years, Duke broke SAP down into eight releases. The first few releases rolled out CRM to the customer and high-risk marketing and commerce. Duke then started replacing each of the four billing systems at a time.
From the beginning, Duke’s leadership knew testing would be a critical component of the project’s success. It was clear that the previous, manual testing approach would not be able to scale with the speed of innovation and accuracy Duke strived for. Streamlining the project would require a testing strategy that would maximize efficiency while minimizing risk to ensure a smooth go-live and superb customer experience.
IT leaders at Duke evaluated the testing market to identify the right solution for faster, safer SAP releases. Ultimately, they determined that Tricentis Tosca’s model-based testing and close partnership with SAP would help them achieve the testing excellence they were looking for: resilient test automation across Fiori, Hybris Marketing, Hybris Commerce, Customer Recommendation engine, SAP ISU, and HybrisC4C.
The team started their journey by auditing the manual test cases they had used in the past and introducing test automation in the highest-risk areas. The emphasis was placed on test cases with high business value that would need to be repeated from one release to the next. This risk-based approach resulted in a regression test suite of 150 test cases that run through 3 testing cycles in various environments for Duke’s weekly SAP releases.
“We didn’t have to rewrite those test cases because we could utilize the customer data for the other jurisdictions within those test cases and then continue to confirm that those components didn’t fail while our manual testers could test the more complicated things that were coming into the system for that jurisdiction,” said Smith.
Today, each SAP release undergoes five different phases of testing — including 250 automated tests weekly. Smoke tests are executed by the IT teams as they deploy changes to ensure critical components of the system are working correctly before passing them off to the business for product and operational readiness testing.
“It’s taking about 32 hours a week, compared to previously it was an average of four hours a test case when done manually,” said Smith. “If you imagine 250 times four hours, you’re talking 1,200 to 1,300 hours just for execution and gathering data, compared to 32 hours in which we’re running those test cases.”
Thanks to Tricentis test automation, Duke’s test cases run unattended overnight and are completed in less than twelve hours – something that could never be possible with the manual testing method of the past. This has also led to earlier defect detection and resulted in higher-quality code being deployed to production.
Duke Energy now has the largest SAP Cloud for Customer (C4C) cloud-based user environment in the U.S. that’s running in a production environment. One early win in Duke’s ongoing S/4HANA transformation project: Duke has already been able to release unified billing for customers based on data generated from their SAP system. This has been a huge success for the business and created the consistent customer experience they were striving to achieve for the meter-to-cash process and customer support.
“We were able to take our first deployment with S/4HANA into production versus having it on some other platform and then having to convert it. We had to shift and adjust, and then actually went in seven months earlier than the original schedule,” said Smith.
“Over the last year, we’ve saved $4.3 million in resource money of not having to do a weekly test bed for regression testing,” said Smith. “We’ve also found 11 defects over the course of the year that, had we not tested in one of our lower environments, would’ve had the potential to go to production. There’s been a lot of savings and value add with doing the automation and bringing it in before each weekly release gets done.”
In 2016, Duke merged with Piedmont Natural Gas (PNG). Next on its testing journey, Duke is building a greenfield to deploy SAP to the PNG environment by July 2024.
“We’re going to pull forward all the learnings that we had from our automation space for Duke Energy, and we will begin an automation space for PNG. We’ve shifted left some and started to look at doing some automation in the conversion space, which we did not do for the Duke side of the house. Then we’ll get into building a regression test bed later in the testing phase,” said Smith.