SAP Test Management Survey Outcome:
Test procedures in SAP Systems
By far the most important test procedure for the community was Function Testing. There was a noticeable lack of test procedures in Integration (testing how the systems interact with each other) – 30% did not conduct Integration Tests. Last on the list was Regression Testing – over 50% of the research participants do not conduct Regression Tests.
Test effort in SAP projects and SAP running Systems
- 30% of the budget in SAP projects is spent on testing
- 20% of the budget in running systems is spent on testing
Common Challenges with SAP Tests
- Not enough resources from the business units
- Significant time spent executing tests
- Lack of automated tests
Bug density in running (productive) SAP Systems
SAP bugs can be very costly due to the sheer size of the SAP installations and the workforce which has to work with the system on a daily basis.
- 11% have several bugs/week
- 24% have several bugs /month
- 35% have several bugs/quarter
Self-Assessment of the Companies
Common factors found in the companies with lots of SAP bugs:
- Their Test Data quality is sub-par
- Their Tests do not correspond with the reality
- Lack of automation
As a Tricentis Consultant I have faced more than a few customers with their own quality issues. Due to the high complexity and diversification of SAP systems in the field it is not so easy to draw comparisons between SAP environments – you don’t want to compare apples with oranges…
A survey out of the SAP community allowed me to reflect on my own customer experiences and compare them to the “general” self-assessment of a group on a larger scale.
It is amazing how good of a fit this self-assessment compares to the needs and worries my customer faced within their SAP projects. The SAP ERP environment has a significant impact on the business for medium sized (and upwards) businesses. Therefore it is easy to understand that the corresponding investments towards Quality assurance are quite significant (30% of the Project costs). But nowadays businesses have to face yet another challenge with their IT ERP environments – Time to market!
Ensuring a high quality of the system especially when integrating further process (or customizing existing ones) into the ERP environment is with “traditional” manual testing, extremely time-consuming and therefore does not allow you to react on changes in market or on internal/structural changes fast. Many companies have been complaining about the rather slow technical SAP deployment over time and SAP has reacted on that issue by introducing SAP HANA.
The development and deployment of SAP HANA will become a strong focus in the industry rapidly, but due to the resulting faster deployment possibilities, the lack of speed in testing will over time even become a stronger issue. As already seen in the survey, most companies worry about (and this coincides with my experience) the lack of time to test, the lack of automation in testing and their inability to transfer the business knowledge from the business units to the test departments.
The most efficient companies in future will have trim their cost for quality assurance down without risking too much and exposing themselves to high cost errors while being flexible and reactive towards the market and their internal needs.
Sounds like an incredible challenge. And yes it is!
A future without a tool that allows you to face the biggest challenges – increase efficiency and effectiveness – is unthinkable for the successful SAP companies of tomorrow.