test automation increase
reduction in test cycle duration
time savings in test data requests (TDR)
As the largest 5G network provider in the United States, T-Mobile has rapidly become synonymous with customer-first telecommunications. Relied upon by over 110 million customers, its emphasis on high quality for every single customer led to the top ranking in postpaid carrier customer satisfaction in 2022.
Balancing quality and quantity has been a focus for T-Mobile for the last 20 years. Maintaining its best-of-breed technology quality amid record-breaking customer growth – the carrier added more than 8 million new customers last year – a tightrope that not many can walk. T-Mobile’s growth is largely thanks to its customer-first approach, which drives everything from flexible plan and product offerings to a user-centered online experience. To support this strategy, T-Mobile’s testing team looked to solidify its functional testing strategy to keep customer experience-compromising defects out of production. Key to this initiative is keeping all testing efforts centralized within one platform for improved visibility and faster go/no-go decision-making.
Raju Chavan, Principal Architect, recognized the need to scale test automation to maintain a high-quality digital user experience for T-Mobile’s growing customer base. The T-Mobile team he supports is responsible for testing 52 applications, supporting multiple business units, including Retail, Digital, Care, Prepaid, and T-Mobile for Business.
“When I came on board, we had virtually zero test automation in place,” Chavan said.
The team evaluated a number of solutions and ultimately chose Tricentis Tosca and its low-code, model-based approach as the best solution for upskilling the company’s manual testers and quickly scaling test automation. One attribute that proved immediately beneficial was Tricentis Tosca’s ability to quickly design and build test cases through its AI-powered, low-code, model-based architecture.
By rapidly designing and reusing these test cases, Chavan found his team could automate numerous tedious small manual tasks, including opening the virtual retail stores automatically, creating test data request tickets, and updating Excel sheets with extracted data for reporting purposes. This ease of use proved its value in rapid adoption among the T-Mobile team, and they found over 300 team members becoming skilled Tosca users within the first year of usage.
The team leverages the test case design feature for rapidly estimating and automatically generating test cases that include test case name, preconditions, test case description, expected results, and verifications, transforming a normally hours long test design process into an easy and scalable process.
However, the team sought to unify test cases within Tosca further. He implemented team-wide policies to create hybrid (mix of manual and automated) test cases within Tosca alongside Tosca’s usual automated test cases to keep his team’s test cases consolidated in one repository. These hybrid tests, containing Tosca automation steps, can be automated outside the manual steps, which usually extend to manual-required verifications. The team also links all manual test cases created outside of Tosca within the Tosca test case repository to keep consistency across the QA organization’s test policies and standard best practices.
Within 8 to 10 months, the team found its testing strategy had achieved 0 – 60% automation throughout an extensive range of tools and applications, including T-Mobile.com and Salesforce.
The team took it a step further and built and modeled over 300 different possible customer business processes (sign up for prepaid service, add a user on an existing plan, change their rate plan) through Tosca instead of manually spending hundreds of hours provisioning for the myriad different possible customer journeys.
In the test planning phase before execution occurs, the team utilizes custom-built APIs called by Tosca’s rest APIs to automatically generate test data, enabling them to streamline the test data request (TDR) time by 50%.
The team leverages Tosca’s risk-based testing ability to identify a minimum number of test cases with maximum coverage that reduces their test scope into the high-value critical test areas, all while reporting a “very low” test failure rate with confidence.
Chavan and his team recognized the potential to unlock further automation advances through Tricentis qTest and Tosca’s native integration. With a middle-ground JSON customization between Tosca and qTest, they can now seamlessly automate Tosca test results into qTest.
Manual, automated, and hybrid test case results can all now be recorded centrally within qTest’s analytics and reporting database. With qTest Explorer’s screenshot functionality, instead of manually attaching a screenshot to qTest, Explorer allowed them to attach the screenshot directly to the corresponding test case result in qTest, alleviating a good deal of frustration.
The ability to integrate with Jira proved another qTest advantage; with one click, the team is now able to view any defect that occurred in a given test throughout their expansive environment. Beforehand, defect reporting took up to ten minutes – now it’s almost instantaneous. With another plugin to their Tableau instance, qTest exports all test data for further inventory and analysis for the T-Mobile team.
For data quality testing, the team added Tricentis Data Integrity to the mix for a critical migration project from T-Mobile’s legacy billing system from Sprint’s 30 million users to its own T-Mobile platform. They undertook the daunting task of validating in-flight data comprising of hundreds of thousands of data points, and after the functional testing flow was completed, they used Tricentis Data Integrity to ensure that mountain of data was validated properly within their database.
By leveraging Tricentis Data Integrity and Tosca, they’ve reduced that test and validation cycle down from two weeks to three days for one of the 300 possible rate progression plans that the team had built (referenced above).
“Data Integrity was one of our heavy lifters in our huge shift from our former billing system to the current system,” reported Chavan.
Overall, the team uses the Tricentis platform to automate a dizzying array of manual tasks that prove more innovative than the next in their creative usage of Tricentis tools. Even when factoring in the robust returns the team has seen using Tosca, qTest, and Data Integrity in their intended ways, the team continues to break ground. They hold industry-grade quality patents in several processes for their use of Tricentis tools for their internal and customer experience and have several upcoming initiatives with which to apply Tricentis tools’ automation power, including integration testing to test APIS, accessibility testing, and security testing.
“We found that relying on one excellent tool that covers 90% of our problems instead of trying to hodgepodge five different open-source tools such as Selenium to cover 100% problems is the best approach,” said Chavan.