testing efficiency gain
tests executed per month
Baloise Insurance is the Belgian branch of Baloise Group, a Swiss-based insurance company focusing on optimal customer service combined with innovative products and services. The company is headquartered in the Swiss city of Basel (hence its name) and provides prevention-, pension-, assistance- and insurance solutions. Its core markets are Switzerland, Germany, Belgium, and Luxembourg. The Belgian Baloise Insurance headquarters are in Antwerp.
Baloise Insurance works through a network of insurance brokers and offers both its brokers and its customers digital access through dedicated MyBaloise portals. Reflecting its mission of providing optimal customer service, the company wants to ensure smooth access and a great customer experience on these portals.
Baloise Insurance had rolled out the Cleva insurance software into its landscape. However, the cost of regression testing was getting out of hand over time. It was no longer possible to test everything manually in an effective and efficient way.
“If we wanted to test Cleva manually, we would have needed at least an additional five full-time people to do that,” said Danny Huijbrechts, System Expert Testing at Baloise and Product Owner for Test Automation at Baloise Insurance. Rather than leaving many changes untested and to mitigate the regression risk, Baloise decided to investigate which test automation tool was the best fit to rollout not only on Cleva, but also on other applications.
Tosca was selected in 2017 from the different tools that were presented by the Baloise implementation partners. “We asked each partner to present just one solution, which they thought was best suited for Baloise,” Huijbrechts recalled. “We wanted the partners to think thoroughly together with us and hence select ‛the ultimate tool’ for us.”
Baloise Insurance decided to choose Tosca because of the vast number of technologies it supported. “Tosca can be used for test automation on any application, whatever development tool is used, or technology is chosen. In the five years that we have been using Tosca, we have not encountered any application that we could not apply Tosca to.”
Baloise also attached great importance to the leadership position that Tricentis takes in the magic quadrant of Gartner and the reports of other analysts. “Our core business is insurance and it is more efficient to rely on the opinion of renowned companies that have a better insight into the market and have already researched and evaluated different solutions,” Huijbrechts explained. “We would never roll out a solution that is not in the right quadrant at Gartner or Forrester. If they say a solution delivers the best return on investment, we follow their advice while taking into account our own specifics.”
Baloise Insurance defines what is automated and is responsible for the ‘why’ – the external partner, for its part, is responsible for the ‘how’ of good test automation, including guidelines and best practices. “Accenture was the integrator that brought Cleva into the Baloise landscape, and they are also the partner that proposed Tosca because they knew it was well suited for both Cleva and other applications going forward. It was definitely an advantage knowing they had already tailored the Cleva insurance application to our way of working.”
Baloise was among the first Belgian customers to use Tosca. “This was certainly an advantage, we received a lot of support from Tricentis and we benefited from the experience of Accenture as a global Tricentis partner.”
The test automation team at Baloise was one of the frontrunners for Agile working at the company. “We wanted to see whether Agile worked at Baloise or not,” said Huijbrechts. “So one big project was chosen as a proof of concept for Agile, and this method was rolled out to various teams of which test automation was one of the first.”
Afterwards, Baloise Belgium started the rollout of the SAFe (Scalable Agile Framework) model, where the System Team contains a dedicated team for test automation. This Central Test Automation Team has three main functions: maintain the test automation tools, monitor the quality of test automation of other teams and, lastly, set up and maintain test automation for other teams.
“We often compare Tosca to an icebreaker that keeps going at a steady pace,” said Huijbrechts. “In Agile, the vision is to move to in-sprint test automation ensuring that you can deliver tested functionality in every sprint. If you want to use test automation in the right way, you need to ensure that tests are reusable and repeatable. This reusability helps you deliver at a consistent, high level.”
“Until recently, we tested mainly UI since that was the area with the highest risk and that gave the highest value for the business,” said Huijbrechts. “We will be focusing more on testing services as these tests are quicker to execute and easier to maintain since a UI is, by definition, more volatile. For low-level test automation we complement Tosca service testing with ReadyApi, which is closer to the technology stack for the development teams.”
Baloise built its own test automation dashboard to regularly monitor how well its test automation is performing and the results are impressive. “We built the dashboard to make our test automation metrics more objective and visible in order to market them to showcase the value of test automation,” said Huijbrechts.
The company supports continuous testing to enable a higher velocity of software releases, and the Test Automation Team is noticing an increase in the defect detection rate, mainly on regression defect detection. “With continuous testing, you detect defects much earlier in the process. Forty-five percent of critical regression defects are found through test automation,” said Huijbrechts.
“The power of test automation only comes when you can run unattended and in parallel,” said Huijbrechts. “We use several servers and we run five to six sessions on each server. That gives us incredible speed in executing tests.” In hindsight, Baloise feels it should have been using DEX (Distributed Execution) sooner. “That would have given us more speed from the start.”
Immediately turning to DEX is only one of the lessons learned that Baloise wants to share with other users. “The power of test automation does not reside in automating your manual tests, since these typically only cover what COULD be tested. The power of test automation is to industrialize and make a step up so you can test all that SHOULD be tested,” said Huijbrechts. “When you test manually, you restrict yourself to the most important cases. When you automate, you can easily run ten times the number of cases, providing a much fuller coverage and increasing quality assurance.” By industrializing test automation Baloise also feels it can build broader tests, checking more parameters in one go than what is possible in manual testing.
Baloise attaches great importance to strict naming conventions. That is an industrialization effort too: by ensuring everyone in development and testing uses the same names for the same functionality, the analysis is performed faster and more accurately. “Just like in standard software development there are systems architects, designers, and developers, you also need someone in test automation who defines the vision and the concrete structure and guidelines that everyone needs to follow. If you maintain that in the right way, you get even more value out of your test data. Test automation requires insight, organizational structure, and control.”
To Baloise, test automation is key in delivering customer experience to internal and external users.