Case Study

Leader in robotic-assisted surgical procedures

Salesforce test automation for robotic-assisted surgery software
66%
reduction in testing time from Selenium to Tosca
200
test cases automated in Sprint +1
35
critical business test cases automated and run daily for "sanity testing"
image

Salesforce test automation with surgical precision

Overview

Software has fueled amazing lifesaving advancements in the surgical and medical device industry over the past decade. For example, in 2018, nearly 1M robotic-assisted surgical procedures were performed in the United States alone—making surgery more effective, less invasive, and easier on surgeons and patients alike.

One leader in robotic-assisted surgery wanted to follow suit and make their software testing more effective, less invasive, and easier as well. The company leverages Salesforce Sales Cloud and Marketing Cloud to accelerate its growth and drive better awareness and adoption of its industry-leading products.

To remain on the cutting edge of technology internally, the company knew it was time to promote their Salesforce Classic environment to the latest functionality offered in Salesforce Lightning. One critical component of that transition was the ability to quickly test each business functionality in Salesforce during the transition, and for every subsequent quarterly release in Lightning.

The company’s implementation of Salesforce Classic included significant amounts of custom code that needed to be refactored for Lightning. It also included many critical integration points with their broader application stack (including SAP). All in all, confirming that these customizations and integrations would continue to work once they moved to Lightning meant more than 300 screens would need to be tested. This required a huge library of test cases. But this testing was not a one-time requirement. After the transition to Lightning, the customer wanted to repeat these tests continuously across future sprints to ensure that any new issues were uncovered as soon as they were introduced. They realized that to achieve the needed speed and repeatability, they needed reliable test automation.

At first, they tried automating their Salesforce testing with Selenium. They built a large suite of Selenium test cases to test their Salesforce Classic screens. However, the maintenance required to update Selenium tests for Salesforce soon raised red flags. These maintenance concerns are summarized in Salesforce’s own notice to developers, “Salesforce changes the DOM in each release. One side effect of all these changes is the impact on test maintenance. Because these improvements modify the Document Object Model (DOM) structure, tests that rely on specific implementation details in the DOM tend to be brittle and require continual updates from release to release.”

Because Selenium tests rely on the DOM, the company knew it needed to find a Salesforce test automation solution that was faster and easier to maintain.

“Continuous testing was needed to facilitate these changes. All the test cases HAD to work and we have a huge regression library. What was automated for one sprint, needed to be functional for the next without additional rework or intervention.”

Challenges

  • Resource-intensive project to test the transition from Salesforce Classic to Salesforce Lightning
  • Custom code previously built for Classic had to be refactored for Lightning UI
  • Integrations to other critical systems needed to be maintained and tested
  • All existing Selenium tests required extreme modification to adapt to the 300+ screen changes
  • Selenium tests required constant maintenance as changes were made from one screen to the next
  • Continuous testing was required as part of the process

Solution

Tricentis test case automation is now used to automate 200 test cases for each Salesforce UI change using a Sprint +1 methodology. Within each sprint, specific user stories are identified, then manually tested to understand what has changed and how within the UI. Once user stories are confirmed, test cases are quickly automated and built into future sprint test plans.

Tricentis’ AI-powered Salesforce engine alleviates the team’s Selenium testing challenges by automatically creating reusable, resilient test modules that work across both Salesforce Classic and Lightning. With this approach, their test cases no longer have to be rewritten for each new version release of Salesforce, significantly minimizing ongoing test case maintenance.

To rapidly scale their Salesforce test automation, the company is using a Sprint +1 methodology. Within each sprint, the development team creates new manual tests for each user story to confirm that the changes introduced in the sprint do not break the company’s Salesforce UIs. Once the manual testing is completed in the sprint, the tests are quickly automated in Tosca so they can be repeated in future sprint test plans.

The company currently has 200 automated Salesforce test cases. These automated tests integrate into the company’s CI/CD pipeline via Jenkins, and testing kicks off automatically with each development change. Automated “sanity tests” run nightly and the results are automatically emailed to the team at 7am each morning. Testers can than review any failed test cases and report code issues back into Jira in time for the 9:30 am standing development meeting.

This process dramatically shortens the testing feedback loop, enabling Salesforce issues to be identified and resolved with extreme efficiency. Using Tricentis helps the company achieve 3X faster testing compared to Selenium, enabling the company to maintain exceptionally high standards in each Salesforce release.

Their next step: integrating Salesforce test automation with test automation across SAP and other connected applications for complete end-to-end test automation.

Product Mix

“We identified the high-risk test cases and ran them daily as sanity tests via Jenkins. This helped us find code issues as soon as they were introduced.”

“Tricentis gives us a lot of confidence because of the ability to run critical test cases every day. We wouldn’t have time to complete this level of testing otherwise. This was a big success. We were running that test suite until the last day code moved to production and we continue running it after the cutover activities as well.

— Manager of Quality Assurance Information Technology Applications

Results

  • Faster-than-expected transition to Salesforce Lightning

  • Test cases do not need to be modified when the UI changes, and they’re simple to update when the underlying process changes
  • User acceptance testing is performed manually once, then captured in automated regression tests for Continuous Testing

  • Automated tests are used as a quality gate for promoting releases