1. HomeArrow Icon
  2. Case StudiesArrow Icon
  3. ITERGO automates critical Citrix performance testingArrow Icon

ITERGO

ITERGO automates critical Citrix performance testing

%

increase in test frequency per year alongside 85% increase in applications under test

default

%

reduction in test design time vs. Visual Studio

default

%

reduced script maintenance with NeoLoad-Selenium integration

default

Company overview

ERGO is one of the major insurance groups in Germany and the rest of Europe. The Group has operations in around 30 countries worldwide, focusing mainly on Europe and Asia. ERGO offers a comprehensive range of insurance, pensions, investments, and services. In its home market of Germany, ERGO ranks among the leading providers across all segments.

ITERGO Informationstechnologie GmbH is part of the global tech hub structure of ERGO Technology & Services Management AG (ET&SM). The focus of its activities is the ERGO Group with its national and international companies. With future-facing and customer-oriented IT solutions, ITERGO makes a valuable contribution to the success of the ERGO Group and is recognized as a benchmark in the market.

Vienna-based Triscon is a certified NeoLoad service partner that focuses on all aspects of performance testing to achieve high speed, scalability, and stability of their clients’ software.

  • Arrow Icon
    Industry: Insurance
  • Arrow Icon
    Organization size: 500
  • Arrow Icon
    Location: Germany
  • Arrow IconProducts:

Challenge

ERGO, one of Europe’s largest insurance groups, was looking to upgrade its version of a business-critical document-classification application delivered over the Citrix protocol. Knowing that there would be major changes with the upgrade and with no previous performance metrics to work from, ITERGO (the primary IT service provider for the ERGO Group) needed to baseline performance and resource consumption of both versions to understand the impact.

Solution

Using the Tricentis NeoLoad performance testing product along with certified service partner Triscon, ITERGO had already realized highly automated, faster, and more efficient performance testing.

Now they wanted to take their continuous performance testing to the next level: automate performance testing for Citrix-virtualized applications and enable fully automated self-service performance tests.

NeoLoad simplifies complex Citrix testing

Testing Citrix virtualized applications is notoriously difficult with hand-coding script tools. Testing Citrix apps is all about the visual outcome (e.g., ensuring that the x-y coordinates are displaying correctly in the user’s window), and you often have to rewrite a script three or four times before you get it right. NeoLoad, on the other hand, uses a visual test design approach. You can see exactly what the test looks like in real time. With NeoLoad’s drag-and-drop, point-and-click approach, it’s one and done — when you’ve finished designing the test, you know it’s accurate and ready to run load against.

“With NeoLoad, the test itself is quite easy, compared to other tools,” said Fabian Seitl, the Triscon performance engineer leading the project.

“We worked with an internal ITERGO colleague to enable him to do the entire test design. He was completely new to Citrix testing, but with NeoLoad it took only a day to enable him to design Citrix tests by himself,” said Roman Ferstl, Triscon founder and performance engineer.

“With NeoLoad, the test itself is quite easy, compared to other tools.” — Fabian Seitl, Triscon performance engineer

The Citrix-virtualized application that ITERGO needed to test was PROKEY, a business-critical document-classification application that streamlines manual entry of tens of thousands of pages of documents every day. Application performance is crucial, as up to 250 ERGO personnel use the application, and any slowdown would erode operational efficiency and potentially impact customer service.

“If the application is not working optimally, the huge amount of documents to be classified and processed piles up,” said Roman.

Benchmarking “before and after” performance and resource consumption

“There have been problems when scaling in the past. That’s why the project management came to us to performance test the new version vs. the previous version. Now, we knew there would be major changes by introducing a new system, so basically what we were doing was baseline test ‘before’ and ‘after’ to see if it performed better (or worse). And in addition to comparing performance, one of the key things we wanted to look at was the impact on infrastructure, whether ITERGO needed more resources,” he continued.

Alexander Hanelt, ITERGO Test Manager, added: “Since the pandemic started, our servers have had to deal with increased load and with business critical applications running on them. It was crucial to understand the additional impact of the upgrade.”

ITERGO was able to benchmark “before and after” resource consumption so that Ops knew exactly how the new client impacted infrastructure — before the app went into production.

“There was a major change on the app — the previous version had some C++ native interfaces replaced by .NET libraries — so we did a comparison test, running one test on the ‘old’ version, and another one on the ‘new’ version to get some benchmarks,” said Fabian. While response times were not meaningfully different, resource consumption was.

“By running such tests, you avoid getting blindsided by problems: there are no surprises, you know about them and can prepare for them.” — Roman Ferstl, Triscon founder

Giving Ops a heads-up

“The Ops people who run these servers thought it was awesome that we did this test because it helped them know about the changes in the resource consumption and prepare for them. Resource consumption of Citrix applications is an important topic for many customers; they need to know whether there’s still enough space in the infrastructure when rolling out new releases. By running such tests, you avoid getting blindsided by problems: there are no surprises, you know about them and can prepare for them,” said Roman.

ITERGO’s journey to fully automated self-service performance testing

Working with Triscon since 2017, ITERGO has been able to transform its performance testing into a modern, highly automated practice. ERGO understood that to retain its leadership position in the ultra-competitive insurance industry, introducing innovative services and ensuring digital performance were crucial. Like every large enterprise, ITERGO had a lot of different testing requirements: both monolithic enterprise applications (like SAP) and microservice-based architectures, both end-to-end testing and API testing, etc. ITERGO realized that if it kept doing things the way it had always done them, it would not be able to keep pace with the accelerated volume, velocity, and variety of software releases.

“We have been able to go from 15-20 test per year for about 5 applications to 200-250 tests per year for more than 40 applications.” — Roman Ferstl, Triscon founder

First step: Eliminate manual effort in performance testing

“The first thing we did was switch to a load testing tool that actually helps you spend less time in script maintenance and script test design by having cool auto-correlation features and frameworks, then automate test design by reusing existing functional tests and having fewer breakable scripts,” said Roman. They moved from Visual Studio to NeoLoad, which immediately reduced test design time by 30-50%. Then they utilized the NeoLoad-Selenium integration to convert functional tests to performance tests with just a click, further cutting script maintenance by 40-90%. “With these two steps, we have been able to go from 15-20 test per year for about 5 applications to 200-250 tests per year for more than 40 applications. And we did this by needing only another two people. Now, we run 10X more tests and cover 8X more applications.”

“We use the NeoLoad-Dynatrace integration to analyze our results in the most efficient way since we don’t have to re-perform tests to do extra profiling.” continued Roman. The Dynatrace-NeoLoad integration is bi-directional, meaning that whatever monitoring metrics are captured by Dynatrace are visible in NeoLoad and whatever performance test metrics are captured by NeoLoad are visible in Dynatrace. Nobody has to jump from screen to screen, cobbling together data from different tools. Root-cause analysis is fast and accurate.

Next step: Fully automate self-service performance testing

“Our goal at ITERGO is to automate performance tests by implementing automated quality gates. We’ve already done this on a small scale: using NeoLoad, Dynatrace & Keptn to automatically evaluate performance tests and enable them to run in pipelines to make automated decisions. We leverage NeoLoad as a collaboration platform. We want the teams to be able to run the tests themselves outside of pipelines by simply hitting a button,” explained Roman.

“This has huge potential. We’re starting with a Dev team responsible for developing OpenShift microservices. Right now, that’s about 10-20 microservices, but we plan to scale up automated quality gates as part of onboarding new microservices — another 240-480 fully automated tests per year.”

By offering performance tests as self-service for Dev teams, ITERGO will get faster feedback earlier in the development stages, where catching problems prior to production is less expensive, easier to resolve, and avoids late-stage delays to releases.

“ITERGO asked how long it would take to implement this kind of automation. With all the ingredients [NeoLoad, Dynatrace, and Keptn] already present, we estimated that it’s only a matter of 2-3 additional days per service to provide automated performance test feedback.”

“Once we have everything set up for them, all they have to do is hit a button to start the test themselves.” — Roman Ferstl, Triscon founder

“We use NeoLoad-Web as the platform to push our approach to performance testing as a service and to get in closer touch with the teams as they are able to look at the results themselves. Once we have everything set up for them, all they have to do is hit a button to start the test themselves (and we would just coordinate which test runs when). This enables them to be completely autonomous to check something on their own. For us, NeoLoad is the central platform for collaboration between the performance center of excellence and the development teams. If you don’t understand the test results, just give us a call and we’ll talk about it. If everything’s green, there’s no need for us to get involved,” Roman concluded.

Tricentis partner: