Top legal and business information provider
How a global research leader introduced DevOps performance testing into their multi-cloud environment
faster test cycles
default
lower AWS testing costs
default
apps migrated to NeoLoad, 100 test scripts
default
Company overview
“Zero-touch” performance testing slashes release cycle time and cloud costs
The leading global provider of legal and business information/analytics needed to solve two parallel performance testing–related problems with one solution. First, they wanted to automate performance testing into their CI/CD pipelines. The company’s existing legacy tool (LoadRunner) was not only expensive; it also hindered their ability to become more agile, reduce test cycles, and promote collaboration between Dev and QA teams. Second, the cost of AWS resources dedicated to testing had to be driven down.
The company’s information network contains over three petabytes of legal and news data, with 128 billion documents from 60,000 sources. Each day the service processes more than 40 million documents and 77 million public records, with more than 1 trillion connections across all content types. With the ever-increasing volume of online research (both number of global users and amount of content) and the company’s innovative powerful data visualization and predictive analytical tools, the company knew they needed to move to the cloud.
Their technology service partner, Cognizant, transitioned the company’s database services to an AWS cloud architecture and shut down their legacy mainframes and servers. With all applications hosted in the cloud, they utilized AWS Auto Scaling, which automatically increases the number of Elastic Compute Cloud (EC2) instances during higher subscriber activity and then lowers capacity when not needed.
However, the company was concerned about how they were using AWS resources for their performance testing. Either there was no capacity for controllers and load generators when needed, or the company was paying for “idle” resources that were not being used optimally (especially during non-testing hours).
Challenges
- Need for cloud-ready solution — all applications have been migrated to AWS
- Lower costs of cloud testing resources
- Better ROI than existing performance testing solution (LoadRunner)
- Reduce manual effort and accelerate test cycles
- Automate performance testing in CI/CD pipelines
- Integration with Git, Jenkins, AWS CLI
- Industry: Other
- Location: Global
- Products:
Solution
Cognizant recommended Tricentis NeoLoad for its faster and easier test design and maintenance, greater automation capabilities, seamless integration with CI/CD tools, and ability to cost effectively optimize cloud resources.
Cloud-resource costs brought under control
The company leveraged NeoLoad’s built-in capabilities and native integrations to dynamically manage testing infrastructure, going from an always-on to an as-needed approach.
When a test is run through NeoLoad, controllers and load generators are dynamically spun up for the test duration and released once the test is finished. The AWS testing resources are automatically provisioned within minutes of executing a test on demand and shut down once the test is complete. With NeoLoad, nobody needs to write lengthy scripts to provision machines, manually connect dynamic testing resources to CI pipelines, or manually change the number of load generators used for a test.
NeoLoad also natively interacts with container orchestrators such as Kubernetes, OpenShift, Microsoft AKS, Amazon EKS, and Google GKE to provision and use load testing infrastructure automatically, on demand.
Further controlling costs, the NeoLoad license is used only when needed. When one team is finished with its test, another team can lease the license to execute its test.
The company realized an 80% savings on their AWS hosting costs with NeoLoad.
We realized an 80% savings on our AWS hosting costs with Tricentis NeoLoad.”
— Automation Test Lead, Global legal and business research company
Tricentis NeoLoad empowers the world’s leading legal and business research company to bring a DevOps approach to performance testing for their 100% cloud-based application environment.
Automated performance testing integrated into CI/CD pipelines
The company realized the benefits of an Agile/Shift Left approach — easier, faster, and less expensive to catch and fix issues earlier rather than later — by integrating automated performance testing into CI/CD pipelines. The performance team developed a “zero manual effort” solution integrating Git, Jenkins, and AWS with out-of-the-box NeoLoad capabilities.
On build deployment, a Jenkins job is triggered, automatically executing a performance test, with test results shared with stakeholders. No manual effort is required to clone the project to the Git repository or map the load generators to the test scenarios. All results are stored in NeoLoad and can be reviewed anytime during or after the test.
Additionally, again with built-in NeoLoad functionality, a second Jenkins job is triggered, creating a new controller in AWS dynamically. The latest NeoLoad project is cloned into the new controller from Git with new AWS load generators created based on the requirements. The test is executed in NeoLoad (with the license being leased based on the number of users and duration, and unleased automatically after test execution), the load generators and controller terminated post–test completion, and test reports are automatically generated to targeted stakeholders’ dashboards.
Through early detection of API performance issues, the company has been able to shorten their time-to-fix on new features and architectures. Utilizing NeoLoad, they now can automatically pass/fail new features based on SLAs for every release or merge.
Results
- Reduced AWS hosting costs by 80%
- Accelerated testing cycles by 84%
- Integrated “zero touch” automated performance testing into CI/CD pipelines
- Migrated 60 applications and over 100 LoadRunner scripts to NeoLoad with no additional effort
- Automated delivery of results analysis and reports
- Scaled multi-cloud approach increased productivity and saved costs
- "With Tricentis NeoLoad, we reduced testing cycles by 84%.” — Automation Test Lead, Global legal and business research company
Migration from legacy tool was not a technical challenge
Any organization switching to NeoLoad from LoadRunner (or any legacy performance testing tool they have used for years) would understandably consider the level of effort involved. How could their performance engineers carve out time to migrate the hundreds of legacy scripts when there already weren’t enough hours in the day? This company took the proven approach of resisting the urge to migrate everything at once but rather focus on only the most critical applications first, then tackle the rest one by one on the fly. What they discovered was that:
- Developing a script in NeoLoad once is no more work than updating it once in LoadRunner.
- The time savings and reduced effort compound over time. Every subsequent script update in NeoLoad is significantly easier and faster.
- About 80% of legacy scripts were actually obsolete and could be discarded.
- Most scenarios (± 70%) could be migrated to NeoLoad in just a couple of weeks.
The company migrated 60 applications, developing/updating 100 test scripts (and discarding about 400 other scenarios that comprised “dead code”), to NeoLoad in a matter of months.
NeoLoad’s ease of use with low-code, rapid test design along with automated script updates and best-in-class CI/CD integration enable QA and Dev teams to cross-collaborate at greater frequency. Faster test scripting and automatic maintenance have resulted in automated continuous performance testing at scale and speed.
Bottom line: Testing cycles have been reduced by 84%