Tricentis Morning Show – Episode 6
Tune in and watch this episode of Tricentis Morning Show to hear expert insight on SAP testing from WienIT, Nagarro, and Tricentis.
Establishing a performance testing strategy is the first and most important step of performance testing. It helps you define:
It’s never possible to test everything, so conscious decisions about where to focus the depth and intensity of testing must be made. Typically, the most fruitful 10-15% of test scenarios uncover 75-90% of the significant problems.
Risk assessment provides a mechanism by which you can prioritize the test effort. It helps determine where to direct the most intense and deepest test efforts and where to deliberately go lighter (to conserve resources for the more intense scenarios). Risk-based testing can identify significant problems more quickly/earlier in the process by helping focus on testing the riskiest aspects of a system.
For most systems, problems related to performance and robustness occur in these areas:
Here is a list of questions prepared by industry expert Ross Collard to help you identify different performance risks:
The answers to these questions will help identify:
Once the functional areas required for performance testing have been identified, de-compose business steps into technical workflows that showcase technical components.
Why should business actions be split into components? Since the goal is to test the performance at an early stage, listing all important components will help to define a performance testing automation strategy. Once a component has been coded, it makes sense to test it separately to measure:
Moreover, component testing supports JMS, APIs, Services, Messages, etc., allowing scenarios to be easily created and maintained. Another major advantage of this strategy is that the components’ interfaces are less likely to be impacted by technical updates. Once a component scenario is created, it can be included in the build process, and feedback on the performance of the current build can be received.
After each sprint, it is necessary to test the assembled application by running realistic user tests (using several components). Even if the components have been tested, it is mandatory to measure:
The testing effort becomes more complex with the progression of the project timeline. In the beginning, the focus is on app quality, then concentrated on the target environment, architecture, and network. This means that performance testing objectives will vary depending on the project’s timeline.
It is imperative that the system being tested is properly configured and that the results obtained can be used for the production system. Environment and setup-related considerations should remain top-of-mind during test strategy development. Here are a few:
It is important that the test environment mirrors the production environment as closely as possible (some differences may remain, which is acceptable). Even if tests are executed in the production environment with actual production data, it would only represent one moment in time. Other conditions/factors would need to be considered as well.
The test plan is a document describing the performance strategy. It should include:
The test plan is a key artifact of a well-designed and executed performance testing strategy, acting as evidence that a team has satisfactorily accounted for the critical role performance plays in the final end-user experience.
In many cases, project teams ensure the delivery of performance test plans as part of feature work during planning and development cycles by requiring them as part of their Definition of Ready. Though each feature story or use case may not require the same level of performance testing, making the thought process a hard requirement for completion leads to better systems and an improved mindset over the end-to-end quality of what the team delivers.
This is the second article in a four-part series focused on practical guidance for modern performance testing:
Part 2 – Establishing a performance testing strategy
Part 3 – Modeling performance tests
Part 4 – Executing performance tests
This blog was originally published in January 2018 and was refreshed in July 2021.