A Complete Guide to API Contract Testing

The current software development era is an era of microservices. The complexities of large bulky parts of the software have forced us to break them into smaller pieces that can be worked on independently and communicate with each other using lightweight protocols. As a result, we increase the overall efficiency of the system by taking control of more minor things that ultimately combine to make the complete software.

But to ensure that this communication never breaks at the user level and we do not invest too much time in testing this, we turn towards contract testing. Its implementation, importance, and working with respect to APIs are what will serve as the core theme of this post.

What is Contract Testing?

Generally, when we talk about a contract, we refer to an understanding between two or more involved parties. It can be a “written” or “verbal” contract, but the type does not matter. What matters is that all the parties understand the terms and are on the same page.

Similarly, when we introduce contract testing, we are testing a contract between two parties. These parties can be any two elements of software design connected to any protocol as per their software design strategies. However, for our understanding and considering the most popular elements, we will take API and its protocols as part of the contract process.

APIs (Application Programming Interfaces) are the best way to communicate between two elements of the software. We do not need heavy intermediate modules to accomplish our job. A simple link with all the data to the backend does our job quite well. An API may be put to work when either client has to send something or receive something from the database or the server.

This process could be treated as one of the most crucial processes in software execution. If an API fails, the client will not see the correct response, or functionality will not work. This will result in business losses.

Keeping the server downtime or some unusual interruption aside, an API may fail in one of the cases:

• Client is not sending the request as understandable by the server.
• Server is not sending the response as understandable by the client.

These two scenarios indicate that there is a lack of understanding between the client and the server. Or in other words, the contract is broken by either the client (also called API consumer) or the server (also called API producer). When we define an error as “contract broken,” we generally refer to the change in specification understanding from both ends.

For instance, if we take OpenAPI specification (which we can safely assume to be the standard today), a single field may be represented as:

{
   "field" : 1
}

Let’s say this is what our producer and consumer both understand for a single value.Now, we take another field with multiple values, and this is how our producer understands:

{
    "field2" : [1, 2]
}

If our consumer is expecting a different approach here, like as follows:

{
    "field2" : 1,2
}

We may say that the contract is broken between the two elements, which are API consumer and API producer in our example.

To avoid such scenarios, we perform API contract testing and ensure that when the end-user is using the product, the contract works without interruption.

Why does Contract Testing Exist?

To understand the existence of contract testing, let’s eliminate it from the equation for a while. Now, we have microservices that are organized and have been assigned their respective tasks. The communication between them has been established using bulky intermediate modules that are stored on the server. What would be the method here that can test the communication between the modules at both ends?

The only practical method that can be used here is end-to-end testing after integrating all the modules. But since integration and end-to-end both are heavy testing modules requiring a lot of time, this can only be done once. If a new change is suggested in the calling protocols, the whole process of combining big modules cannot be performed again. Apart from time, we may encounter the following issues if we take the integration and end-to-end testing path:

• The integration/ end-to-end testing is extremely slow in development and returning results, especially if we need to test communication modules.
• System maintenance becomes harder as multiple modules are involved, and all of them have to be kept at their best for good testing. Scaling is also an issue when so many components are involved.
• Since the involvement is of multiple modules, sometimes, a difference could be the result of another module’s fault instead of the one we are testing.
• Debugging becomes a hectic task when so many modules are in play. It may also happen that you are debugging an error due to a module apart from the one you are testing. Finding this could take a lot of time.

To avoid so many problems by choosing the end-to-end integration way, it was more beneficial to choose a method that can focus just on one part: communication. It helps us understand where the problem is and completes things quickly.

Benefits of Contract Testing

So, what benefits will a tester enjoy when they go for contract testing?

• Contract testing helps in the smooth integration of microservices and their testing.
• The maintenance part becomes easier and more lightweight.
• Testers can implement and experiment with a large number of APIs when the system is so light.
• Contract testing helps focus on a single module at a time. To test the contract for module A, we need not integrate it with other modules but can test it independently.
• Debugging becomes easier and faster with API contract testing.
• It is easier and feasible to scale up when the software is scaling with time.

In brief, with API contract testing, we can skip end-to-end and integration steps and just focus on what needs to be tested. It brings a lot of additional benefits along with the ones mentioned above, as you might already be aware of as a tester.

When to Do Contract Testing?

Contract testing is an efficient method but can only be applied when you are working on modules that need to communicate with each other. In our case (and the majority of products), this communication is executed using APIs. They make the microservice architecture one of the most reliable solutions to work efficiently in software development.

Once the tester has identified this, they must establish a contract by settling on certain standards or specifications. For instance, in API contract testing, we can opt for OpenAPI specification. If there can be a deviation or more than one possibility of acceptance by the client, contract testing may not be a good solution here.

After this, we determine our objectives to be achieved from API contract testing. This may depend on the tester and the project they are working on. For reference, we can chalk out standard objectives as achieved by a majority of organizations to perform contract testing:

• When complete software development is not yet achieved, and we need to test APIs or other communication methods.
• When service configuration defects need to be exposed.
• When consumer workflow diagnosis is required.

If you have experienced a certain situation where contract testing has rescued you and helped you in the process ahead, let us know in the comment section.

How to Perform API Contract Testing?

Once we have established that our project is perfect for API contract testing, what are the three major steps we need to take to move forward?

Planning – Making a contract

The first phase of API contract testing starts with the planning phase. In this phase, the testers define the role of each team and their tasks accordingly. We focus on integration capabilities and how the communication between modules will be mocked for better testing. The planning phase may include a high-level understanding of each step the team will take.

In this phase, we also separate the testing parts associated with the consumer from that of the producer. This may include settings, specifications, and testing materials for each module.

Documenting the contract

The next most important part is to document the contract. Considering that different teams work on different modules, an understanding between them regarding the correct use of APIs needs to be established. This is the part where we call the “understanding of the contract” is in the process.

Documentation generally starts and floats from the API producer side because they construct the APIs. The documentation requires the teams to document everything they find worthy of contract, including the specification standards they use. This documentation, when passed to the consumers, helps them write code that can be implemented on the client side, considering all the logical functions and methods described in the doc.

Tool Selection

Once the theoretical part is over, we need a good tool for API contract testing. The primary characteristics we need to look for are given below:

• The tool is easy enough to be understood by people of varying expertise across the teams.
• The tool provides all the necessary important functions required in API contract testing.
• The tool can test the consumer side as efficiently as the producer side. After all, we don’t want to work on two different tools for both ends and increase our work.

While there may be a few other tools in this domain, Testsigma is one of the best ones to start with. Testsigma is a cloud-based automation testing platform with all the major testing fragments built into the system. It uses English as the medium of communication between the tester and the testing framework. As a result, someone who has just started their testing journey can also construct test cases quickly.

Apart from tons of helpful integrations and AI-based systems, Testsigma provides the following features for the testers:

• Support for parameterization for testing large sets of data without coding multiple times.
• Helps in storing and reusing the API response data automatically.
• Validates actual responses with expected ones in different modes.
• Codeless testing to wrap up testing from start to end without writing a single line of code.

For a beginner or an expert, Testsigma can be a good point to start your API contract testing journey. It is free to use and just a sign-up away from you.

Validation and confirmation

The last part of performing API contract testing is the validation and confirmation of the results and APIs. This will establish whether both parties confirm the contract or not.

Sometimes, this stage can be performed in the tool itself. But if it doesn’t, the manual way is the way to go through, and it won’t take too much of your time.

When Not to Do Contract Testing?

The advantages and speed brought into the testing cycle by contract testing are tempting enough to experiment with it in all scenarios. But can we perform contract testing anywhere?

The answer is that by the nature of contract testing and its foundation, it is not safe to apply it in each scenario. In these situations, we should turn back to end-to-end testing. While it may depend on the tester to adopt this testing where they want, the following scenarios are some from the list where we cannot (or actually should not) do contract testing.

Mocking

The first one on the list is mocking APIs when they have not been completely developed. Mocking in API testing is imitating the APIs that are planned for development. Since they are not yet developed, contract testing cannot ensure that they will produce the same result in production as they are producing with the mock APIs.

This is because contract testing needs strict specification understanding between both sides. When we mock the tests with APIs, we are not yet sure that the same specification would go into production later. Anything can change as they are still in development. So, performing contract testing in such scenarios is not a good idea.

Requirements beyond contracts

Just going by the definition of API contract testing, it can provide confirmation that the producer and consumer are on the same page when it comes to API and its standard protocols.

However, it is also to be noted that the work of contract testing finishes just here at APIs. For requirements beyond contract testing, testers may not find this method helpful at all. Besides, it is not a good idea to forcefully implement scenarios not meant for contract testing. This may include functional testing of the functions that wraps up the APIs or user interface testing, etc.

Data-related scenarios/rules

API contract testing focuses on testing the communication between the involved modules. This is achieved through endpoints and confirming the understanding of the specification and standards between them.

However, APIs may carry much more things that are beyond endpoints. For instance, “if a user provides XYZ input, the data of API should contain ABC” is a rule that must always be true. This cannot be tested with API contract testing as it goes beyond its method of work.

For scenarios and rules that include data, API contract testing should be avoided.

REST API Contract Testing Tools

There are a lot of options for REST-based API contract testing tools to test APIs either on the cloud or directly from the local system with the application plugged in. A few of them are:

• Testsigma
• REST Assured
• Karate
• Postman

If you have any other tool that you use for REST API contract testing, let us know in the comment section.

Read here about Consumer Driven Contract Testing.

Wrapping it up!

When an API consumer and API producer both settle on certain protocols and specification standards that will be used for communication, we say a contract has been established. To ensure that this contract is “actually” established in the practical design and will work on deployed software, we perform contract testing. This method is based only on the specification testing part for the contract-driven design scenarios in the application.

On the other side, API contract design can sometimes give a perception that it can help us analyze much more than just communication, such as API designs, functionalities on modules, UI, etc. This is a common question, and hence with this post, we tried to not only solve this and draw a line between areas where contract testing can be used and where it can’t be but also, along with it, we dived into the concepts to understand the actual role of contract testing in today’s microservices-driven world and the benefits we can reap when we opt for it.

Concluding with the tools that can help you, I hope this post serves as a good guide for your API-driven software and projects.

Thank you for giving this post your valuable time. 🙂

Frequently Asked Questions

What is an API contract example?

An API contract means that the API producer and API consumer both understand the specifications and API-related guidelines so that there are no ambiguous outputs when an end-user operates the software. An example of an API contract could be as follows:

Let’s say Testsigma has a user base and an internal API can fetch data for it or post some data on it. A user object could be as:

{
  id: integer
  username: string
  email: string
}

To get the details of all the users:

{
  users: [
           {<user_object>},
           {<user_object>},
           {<user_object>}
         ]
}

To get the detail of a single user:

Method: GET/users/:id

id – Required parameter. [Integer]

Headers to use

Content-Type: application/JSON

Authorization: Bearer <OAuth Token>

Success Code: 200

Success Response: { <user_object> }

Error Response:

1. 404 – { error : “User not in database” }

2. 401 – { error : error : “Unauthorized to make this request.” }

Similarly, there are a lot more endpoints with each description for the consumer from the producer’s side. API contract testing helps verify these strict specifications at the client’s end.

What is OpenAPI contract testing?

OpenAPI is a set of standards describing various actions for RESTful web services and is used popularly across the globe. A contract testing done on software that uses OpenAPI standards for specifications is OpenAPI contract testing.

What is contract testing in microservices?

Microservices are smaller but independent units of software that work on certain specified tasks. They help us isolate pieces of code, focus on specific things at a time and quickly debug the error. To make the software work, all these microservices need to communicate with each other. This is popularly achieved through APIs. When microservices are used and API contract testing is done to test their communication part, it is termed contract testing in microservices.

Is contract testing an integration test?

No. The most fruitful benefit of using contract testing is that we can dodge the integration testing road that was popular earlier. If only the communication and understanding of APIs are required to be tested, integrations just waste a lot of time and make things harder. Contract testing gives us an alternative to integration tests and helps wrap things much more quickly.

Is contract testing the same as API testing?

A contract is just the understanding between the API consumer and the API producer. When our tests focus on just this part, we say we are performing API contract testing. Anything beyond this involving APIs comes under API testing. You can read a quick guide on API testing to know more.