1. HomeArrow Icon
  2. Case StudiesArrow Icon
  3. Quality assurance in action: The Scottish Government's transformation of social security for a better citizen experienceArrow Icon

The Scottish Government

Quality assurance in action: The Scottish Government’s transformation of social security for a better citizen experience

Company overview

When the Scotland Act gave the Scottish Government control over social security benefits in 2016, the IT department knew it had to build a complex application that allowed for both case workers and citizens to consult and manage social benefits. The Scottish Government wanted to offer its citizens the best experience from the systems they use to interact with them. Tricentis NeoLoad and Tricentis qTest play a major role in assuring application-wide quality and performance for the core applications of those systems, ensuring a smooth interface between government and citizens.

The main application, Social Program Management (SPM) is used by over 2,000 government staff in different locations to handle the processing and approval of benefit requests. They use SPM to submit applications for benefits and take these applications through a long process of eligibility and verification checks before they can be approved. Before any new benefits are issued, the entire flow needs to be tested to ensure good response times, as staff are sometimes filling out the forms while they have a citizen on the phone. The online load needs to be simulated with the anticipated volumes during the test to see how the application behaves during real usage. And the same applies to the citizen-facing Adobe Experience Manager (AEM) portal where beneficiaries submit applications online. The application landscape at the Social Security Services also contains the integration between SPM and other government agencies, such as the Scottish Courts and Tribunal Services, and third-party services. These applications are all exchanging data through Application Programming Interfaces (APIs).

Challenges

  • Complex application landscape with multiple integrations
  • Stringent SLAs for load times of applications
  • Constantly changing demands
  • High volume of tests impossible to manage in spreadsheets

With that broad of an application landscape, test management is key in keeping control of all the testing that is going on. “When we launched the first benefits, a lot of the management was done through spreadsheets,” recalls Jeff Scodellaro, Test Manager at Scottish Government. “But nowadays there are just too many delivery teams, too many test teams, too many sprint teams, and too many development teams. We would really struggle without a tool like qTest. I would say that qTest is set up in much the same way that we are set up as a testing organization and a test team. Having a tool that is adaptable to the way we work definitely influenced the choice of qTest.”

At the heart of quality assurance

qTest is one of the main tools that are used across all the different teams in the Social Security Services of Scottish Government: in-sprint testing, outside-sprint testing, test automation, regression testing teams. “qTest is pretty high up in the grand scheme of things,” said Scodellaro. Business users such as product owners also use qTest to come in and review the tests. “We use qTest as a communications tool to advise teams when things are ready and what they can potentially use in their scope.”

And, of course, qTest also serves as a reporting tool and an organization tool. “qTest allows you to see everything in a really nice way. You can review things easily and tag people into things that you want to see updates on.” Scodellaro is also delighted with the traceability that qTest provides.

qTest is closely integrated with both Jira and the different test automation tools that are in use. The test teams raise defects in qTest and have them go directly into Jira, and the other way around. qTest fits well in the Agile way of working at Scottish Government.

“In my team, I plan almost on a sprint level for testing, where I’ll define all the actual individual stages for our team. After putting the plan dates into qTest and linking that to the requirements, I use that to drive the way we set up the test design and test execution, so that we’re executing in stages rather than everything at once,” says Scodellaro. “If people follow the same quality processes, qTest actually allows you to keep a pretty clean structure for testing. It allows you to define how things are set up, how your folders are constructed, and what’s linked to those.”

Currently, qTest is being used by around 200 people, mainly internal staff, but also contractors, testers, and external service providers. “qTest has allowed us to execute things to the right status, so that a pretty high-quality standard is maintained across the testing. It gives us the ability to manage testing efficiently, see where the gaps are, and where things could be improved,” Scodellaro concluded.

Short response times

With thousands of internal staff using the SPM application, having good response times is critical for employee experience. Before the Scottish Government turned to NeoLoad, the performance testing teams used an open source tool, JMeter. But that tool could not handle the complexity of the application, which had lengthy flows that were difficult to script using the open source tool. “We looked at several tools,” says Iain Page, Performance Test Manager at Scottish Government, “but NeoLoad was the best fit for what we needed. Applications that we could not script before with NeoLoad, we could script them. And the tool feature that you can use for the correlation of scripts is really good. Once we had that in place, we could get scripting done way quicker.”

Other requirements that the Scottish Government was looking for included in-built monitoring and the capability to store all the scripts in one place. “Comparing capabilities versus cost, NeoLoad came out as the best solution,” said Page. Scottish Government wanted to get away from open source tools because of the lack of support these tools offer. They find Tricentis quite supportive, following up quickly on support tickets – although these support tickets are fairly rare.

Most testing is done on the central SPM application, which is web-based, but besides the testing of User Interfaces (UI) there’s also a lot of API testing going on for the components that integrate with

SPM and applications that trigger batch activities in the evening. “We have a Service Level Agreement (SLA) with the users of three seconds of response time, and if we do not get the agreed performance SLA, we raise that concern with the development team and provide evidence from the NeoLoad test results to support the test observations,” said Page. And the same goes for issues detected with the digital portal and some of the batch processes that run too slowly. “To document our findings, we use logs and other information that we get from NeoLoad results. Based on our feedback, we see development take that onboard and we have been able to push through improvements that way.”

“NeoLoad is a suitable fit in terms of ease of use, when recording, updating, and validating scripts, generating varied test scenarios based on test requirements, creating data provisions for testing purposes, and generating NeoLoad test reports with the collated results.” The overall tool features have allowed the team to turn tests around in a more efficient and timely manner, when compared to the open source tools initially adopted in the Scottish Government.

The built-in framework feature is used frequently during release test cycles, reducing time to correlate new and existing scripts when applying the framework. This feature has dramatically reduced updating scripts during program release cycles.

“The learning curve for the test team to use the NeoLoad tool has not been too much of a challenge,” said Page. “This is due to the useful, built-in help library which the tool provides. As a team, we also found the NeoLoad UI is easy to understand and very user-friendly. There is still a learning curve in terms of all the features the tool has to offer, something the team are constantly exploring further while the tool gets used.”

Results

  • Over 2,000 tests scripts per project managed through qTest
  • 25-30% increase in test frequency per year
  • 50-60% time savings in test design times
  • 70-80% increase in applications under test
  • 90% faster testing compared to previous tool

Raising the bar for quality assurance

Using NeoLoad and qTest has definitely raised the maturity of quality assurance at the Scottish Government. “We are constantly getting more questions from different teams to run new tests, even in early development and for in-sprint testing,” said Page. “It’s a good thing they are asking for testing earlier in the pipeline. So we get questions both from the functional teams and from the operational acceptance teams.”

Scottish Government recognizes it is not yet using NeoLoad and qTest to their full potential. “There is so much that the tools can give us that we have not even looked at using. There’s definitely scope to use the tools in a better way, be it the functionalities that we are not yet using or using the tools in a clean and consistent way.”