An Introduction to sanity testing

Software updates are a common element in the life cycle of any application software. Modern software applications ship features more frequently. Sometimes, new features ship with bugs in existing, crucial features that hinder the end-user experience of your application. In some cases, this can even lead to a complete block for your users to use your application.

Sanity testing gives you the ability to ship new features without breaking existing ones. In this post, we’ll understand what sanity testing is and why it’s important. We’ll also explore how it stands apart from other testing methodologies.

What is sanity testing?

Sanity testing is a way to test your software on a surface level and validate that any recent code changes haven’t changed an existing behavior or disrupted any core functionalities. In sanity testing, we only try to ensure that the basic operations work as intended after the updates, without doing rigorous, exhaustive end-to-end testing.

Sanity testing is like a regular health checkup, where you’re not diving into specific blood tests or scans but rather making sure that everything like your pulse and blood pressure, seems normal.

Sanity testing is a subset of regression testing. After receiving the software build, sanity testing is performed to ensure that the code changes introduced are working as expected. This testing is a checkpoint to determine if testing for the build can proceed or not.

– Economic Times

Sanity testing is a way to test your software on a surface level and validate that any recent code changes haven’t changed an existing behavior or disrupted any core functionalities.

Sanity testing vs. other testing types

Even though sanity testing is not as intensive as other testing types, it’s still sometimes confused with other testing types, such as smoke testing, integration testing, etc. Let’s understand some crucial differences between sanity testing and other related types of testing commonly confused with sanity testing.

Smoke testing vs. sanity testing

Both sanity testing and smoke testing have different purposes and use cases. Smoke testing is a broader type of testing that covers multiple facets of the software. Sanity testing, on the other hand, is more narrow, focused, and targets only specific areas after smaller updates have been pushed.

Both testing methods solve a common goal—ensuring that major functionalities work as intended. However, they differ extremely in their scope. This difference further translates into the human resources and the steps required to perform these testing types.

Sanity testing vs. integration testing

Integration testing is about understanding how different modules of the software interact to ensure smooth functionality.

Sanity testing is surface-level testing and hence doesn’t explore these interactions with as much depth as integration testing does. Instead, it merely verifies that the critical functionalities of the individual modules are working fine after the updates.

Where does sanity testing fit in the overall testing process?

Sanity testing is generally performed after software has gone through a set of minor changes or fixes. It acts like a quick gatekeeper and helps developers, QA and test engineers understand if more rigorous and intensive testing, like regression testing or integration testing, is needed.

Sanity testing acts like a quick gatekeeper and helps developers, QA and test engineers understand if more rigorous and intensive testing is needed.

How sanity testing works

Let’s now dive deeper into the exact process by which sanity testing works. A good prerequisite for any sanity testing is a list of features that constitute the core functionality of a software application.

A typical sanity testing process includes the following steps:

1. Identify the modified features or changes

In this step, all the updates or features that are modified or changed are identified, isolated, and most likely documented. This helps in establishing the scope of sanity testing. This step also acts as a check to understand if sanity testing is applicable for this set of changes.

2. Perform targeted tests

Once the changes are identified, targeted tests are performed to ensure that these changes haven’t affected any core functionality.

3. Validate the changes

At this step, either the build is approved to be stable enough for user release or it’s passed onto the required team for more comprehensive testing.

Who is responsible for conducting sanity testing in a software project?

Typically, the QA (or quality assurance) teams are responsible for conducting sanity testing in a software project. Dedicated test engineers can also perform this testing. In small organizations and start-ups, developers should perform sanity testing themselves since they’re most aware of the scope of their changes.

Manual vs. automated sanity testing

Sanity testing can be both manual and automated, depending on how many core functionalities a software application has and the level of complexity involved in those functionalities. In manual sanity testing, QA engineers quickly check the core functionalities and evaluate if they’re working correctly, whereas automated sanity testing uses a set of scripts or automated testing tools to verify the core functionalities.

When should sanity testing be performed?

Generally, sanity testing should precede shipping any changes or fixes. Every time a build undergoes minor code changes, sanity testing can be performed. Since it is very limited in its scope, a high frequency doesn’t drain a lot of time and resources.

• Ideal scenarios for sanity testing include:
• After small feature changes
• Minor bug fixes
• Frequent updates in the environments
• Changes in the dependencies versions

Some of the less suitable scenarios for sanity testing include major application revamps, additions of new integration features, a completely different feature built in isolation to the core functionality, etc.

Practical examples of sanity testing

To understand the scope of sanity testing and how it can be advantageous, let’s look at some practical examples or use cases where sanity testing can be effectively employed.

Mobile app update

Consider an example where you’ve updated a login feature in your mobile app. A quick sanity test would include performing the following checks:

• The user can log in
• The user can log out
• Profile information is shown correctly

Since these checks are small and extremely quick to perform, manual sanity testing would be suitable here.

E-commerce platform

Now consider a scenario where a bug fix for the checkout page of an e-commerce website is being shipped to the users. Sanity testing here would involve verifying the entire checkout process. This includes adding items in the cart, updating those items from the cart, making payments—and therefore testing payment gateway integration—and finally order confirmation.

A manual sanity testing would work here as well, but a script that performs the above checks would be the most efficient way to sanity test such an application every time a build is shipped that involves changes related to the checkout page or the cart functionality.

Sanity testing allows developers and testers to focus on more meaningful testing activities by filtering out issues with builds early.

What are the key benefits of sanity testing?

Some common advantages of sanity testing include:

Speed and efficiency

It helps in quickly aligning on if further testing is required, and it doesn’t drain significant bandwidth and time.

Cost effectiveness

Resources usually spent on exhaustive testing on larger and unstable builds are saved, making sanity testing extremely cost effective.

Improved productivity

Sanity testing allows developers and testers to focus on more meaningful testing activities by filtering out issues with builds early.

What are the cons of sanity testing?

Since sanity testing only covers specific features and core functionalities, it may miss broader issues. It also relies on the accuracy of testers to correctly identify essential features, the scope of the sanity testing, and how those features are connected together. It’s not a replacement for detailed regression or integration testing, given the tightly defined scope.

Sanity testing is the first line of defense that protects you from unwanted breakages after small tweaks and fixes.

Conclusion

Sanity testing is the first line of defense that protects you from unwanted breakages after small tweaks and fixes. By readily identifying if an update is stable, it can save countless hours and valuable resources and ensure a smooth deployment while maintaining the quality of your application.

Performing sanity testing also gives you a sense of confidence and helps you make more informed decisions around avoiding rework on builds, enhancing the agility of your software delivery process.

Siddhant Varma wrote this post. Siddhant is a full-stack JavaScript developer with expertise in frontend engineering. He’s worked with scaling multiple start-ups in India and has experience building products in the Ed-Tech and healthcare industries. Siddhant has a passion for teaching and a knack for writing. He’s also taught programming to many graduates, helping them become better future developers.