Contract testing: What it is and how it works

The software testing field is huge: there are a plethora of software testing tools, techniques, and methodologies. Below, we’ll cover a specific type of software testing you might not be familiar with: contract testing.

Contract testing is a way to ensure smooth and stable communication between services that communicate with each other via some public interface, also called “contract,” hence the naming. This form of testing has become increasingly relevant.

Contract testing has become increasingly important because of the proliferation of APIs and microservices. Yes, they should be independent from each other, but they need to have some degree of interaction with each other in practice. And that means the contracts between them must be as stable as possible.

What is contract testing?

Contract testing is a form of testing that enables teams to define a contract between two services and then verify whether they adhere to the contract. You can consider it a subtype of API testing.

To understand what a contract means in the context of applications, it is very similar to a real contract. Contracts define the rights and obligations of two parties to ensure a smooth relationship (generally a commercial transaction) between them.

The same is true for programming contracts. For instance, a web API could state its contract as follows:

If you commit to sending a GET request to my /api/v1/users endpoint, I commit to sending you back two things: a JSON payload containing a list of users (with fields username, email, and create_date for each one), and a 200 OK status code to let you know everything worked as planned.

If  API started giving back a different payload or a 500 status code, that would be a breach of contract.

Contract testing matters when you have services talking to each other and you need to ensure that communication happens in an efficient and stable way.

Importance within the development life cycle

Contract testing matters when you have services talking to each other and you need to ensure that communication happens in an efficient and stable way. However, contract testing can have benefits even in earlier stages of the development life cycle.

Let’s consider a scenario. You’re developing an application—a microservice or API—that needs to communicate with another service that hasn’t been created yet. This new service belongs to another team, and you can’t know for sure when it’s going to be ready.

You can’t simply do nothing and wait until the other service is ready, because that would make your own service’s development late. What you can do is talk to the other team and agree on a contract for the integration between your two services. Then, you can use a mock implementation (i.e., a “mock”) of their service that adheres to the contract, and use contract testing to test the validity of your mock.

Benefits of contract testing

You’ve learned the definition of contract testing and what role it plays in the software development life cycle, but now let’s cover the benefits of this form of testing.

What Is the Point of Contract Testing?

The main point of contract testing is to act as a form of safety net regarding the stability of the interface/contract of services that need to talk to each other. If you have a service that is consumed by third parties, every time you release a new version, you take the risk of breaking your contract. Breaking the contract brings consequences when it comes to the reputation of your brand and the damage in the relationship with your clients. There are also potentially severe financial losses that can be caused by broken integrations.

If you have a mature contract testing approach that alerts you you’ve broken a contract before you release your new version, you can then decide to either:

• Abort the release, or
• continue with the release, but explicitly version it as a new major release that breaks compatibility

Main benefits of contract testing

Here is a list of some contract testing benefits:

• Reduces the likelihood of breaking contracts, ensuring communication between services works in a reliable way
• Facilitates semantic versioning by alerting teams when a change breaks backwards compatibility
• Enables independent/parallel development of services
• Reduces test environment costs, because contract tests don’t require full end-to-end environments

How contract testing works

Let’s now cover how contract testing works.

• Provider: A service that exposes data or functionality to other services
• Consumer: The service that consumes what the provider exposes

After the parties are defined, they need to agree on a contract, and there needs to be some definition of this contract. For the definition, there are a number of possible solutions available: OpenAPI (Swagger), JSON Schema, GraphQL, or a custom format tailored to your needs.

The “workflow” is:

• The provider generates and publishes its contract definition
• The consumer writes contract tests against said definition, to ensure it can correctly interact with it
• Optionally, the provider can write its tests against its contract and add those tests to its pipeline to ensure backwards compatibility

This process ensures reliable communication between services while allowing them to evolve independently.

How can contract testing be integrated into a CI/CD pipeline?

Should contract testing be integrated into your CI/CD pipeline? In general, yes, but there’s some nuance to it.

Here’s a quote from Martin Fowler, author of Refactoring and chief scientist at ThoughtWorks:

A failure in a contract test shouldn’t necessarily break the build in the same way that a normal test failure would. It should, however, trigger a task to get things consistent again. This may involve updating the tests and code to bring them back into consistency with the external service. Just as likely, it will trigger a conversation with the keepers of the external service to talk about the change and alert them to how their changes are affecting other applications.

Martin is talking specifically about tests from the point of view of a consumer. In other words, you are running contract testing to test your “fake” implementation of an external service against the contract definition provided by them, which they keep up to date. A breakage here likely means they have changed their services. Now you need to change your code to ensure your application behaves correctly.

On the other hand, you should probably be much stricter with failures from tests you run against your contract definitions. That means that you’ve broken the contract you defined with your clients and committed to honor. It makes sense that these failures should result in a broken build.

When to use contract testing

The rule of thumb is that you should consider adopting contract testing when the landscape is complex enough to warrant it. For instance, the following scenarios probably shouldn’t need contract testing:

• You don’t have any services that are consumed by third parties
• You have a service exposed to other teams, but its interface is stable and rarely changes
• The organization you work at is small enough that it’s feasible to talk to all your clients and agree on contract changes before they happen, so no client’s application gets broken due to the changes

If, on the other hand, you have lots of services consumed by third parties, each of which has a complex and volatile interface, and your services are exposed to a big enough number of clients, then contract testing is something you should seriously consider.

Conclusion

Contract testing is essential in a world of constant communication between services. It helps ensure backwards compatibility, facilitating semantic versioning and making parallel development possible.

This post was written by Carlos Schults. Carlos is a skilled software engineer and an accomplished technical writer for various clients. His passion is to get to the bottom (the source) of things and captivate readers with approachable and informative technical content.