Case Study

Dell ISG

How Dell ISG unified testing across diverse DevOps teams and toolsets
50+
product teams with 15K+ people
20+
test automation frameworks
6
months for initial rollout
devops invented

“We were able to take 20 different test case management systems, all working differently, and consolidate them into a single test case management system. This allowed us to achieve our goals around sharing intellectual property, increasing engineering mobility, and fostering collaboration from a test execution and test management perspective.”

Adam Arakelian, Director of Software Engineering at Dell ISG

How Dell ISG unified testing across diverse DevOps teams and toolsets

Challenges

Coordinate test management and execution across divisional product teams with dramatically different methodologies and toolsets and different customer offerings.

Solution

Tricentis qTest

Collaboration through abstraction

Background

Startup-style agility is key for continued innovation and market leadership, but this isn’t easy when you’re completing the largest technology merger in history. Dell ISG’s Storage division was confident that improving collaboration could help them boost innovation while maximizing efficiency and upholding stringent quality standards.

30+ product teams were building, testing, and delivering applications using different methodologies, terminologies, pipelines, and toolsets. Teams meant different things by terms like performance testing and integration testing, and they were completing them with different tools and approaches selected to meet each team’s unique needs. With more standardization, team members could reuse others’ work and even float from team to team—enabling them to complete projects faster. 

Abstraction was key for achieving this without disrupting delivery or homogenizing each team’s unique way of working. They analyzed the elements of success from a high-level philosophical perspective, then crafted an abstract CI/CD/CT architecture that specified what activities to address (source control, requirements management, test management, etc.) without prescribing low-level implementation details on how to complete them. In theory, it’s open to all existing tools and methodologies. But the devil is in the details. How do you make something like reusable, collaborative testing a reality across a division with 20+ test automation frameworks?

Challenges
  • Different teams had different interpretations of what testing best practices required
  • Teams developed vastly different methodologies suited to their unique cadences and deliverables
  • Needed to integrate testing consistently throughout a CI/CD/CT DevOps pipeline with GitHub, JFrog Artifactory, Atlassian Jira—all orchestrated via Apache Kafka
  • Teams invested significant time and resources in building test assets for 20+ different test automation frameworks
Cloud's Continuous Testing Platform

Adam Arakelian

“Tricentis is not just a vendor. Tricentis and Dell ISG forged a strong partnership from the first day we started working together. They understand our strategy; they’re helping us build an ecosystem. At the end of the day, they support us just like an extension of the Dell team.”

Adam Arakelian, Director of Software Engineering at Dell ISG

Solution

The Storage division’s DevOps group decided to kick off the initiative by tackling test management and test orchestration first. After an extensive search, they selected Tricentis qTest. No matter what methodology (e.g., Scrum at scale, Kanban, Waterfall, SAFe) or test automation framework (e.g., home grown frameworks, Robot, Selenium) a team is using, their tests can be managed and executed in a consistent manner via Tricentis qTest. Real-time test execution results from every tool are always available in Jira, and requirement updates are immediately available when working with any test via qTest. Moreover, having a centralized test repository helps teams quickly identify and leverage relevant test assets—enabling them to shift resources from duplication to innovation.

“qTest easily accommodates variations across each team and project while standardizing our test assets, test data, and our overall test process,” explained Arakelian. “Different teams can share tests and even run each other’s tests without even thinking about what test automation engine is used behind the scenes.”

Following impressive success in the Storage division, this effort was uplifted to the larger Dell ISG business division with ~15K people. Dell ISG is also on the path to making Tricentis Tosca “no-code/low-code” test automation the recommended option for all new teams and projects with the goal of extreme reuse through highly modularized and resilient test automation.

Results
  • Accelerated mean time to delivery across teams
  • Reduced the costs of creating and maintaining duplicate tests
  • Enabled more thorough testing of integration points
  • Increased team and divisional agility + throughput
  • Framework made “shift left” testing accessible as well as unavoidable