Estimate project
← All posts

Microservice Testing with Contract Tests

A monolithic application is fairly simple to test, but microservices make testing noticeably harder. Fortunately, every challenge comes with a pattern, and here that pattern is contract testing. See how it makes testing microservices easier.

14 min read
JavaMicroservicesTestingContract tests

A monolithic application is fairly simple to test. We write unit tests, integration tests, and most likely end-to-end tests, while trying to keep the testing pyramid in the best possible shape. Microservices, however, make testing a bit more complicated. What really sets them apart is communication, which lies at the heart of a microservice architecture. Fortunately, every challenge comes with a pattern, and here that pattern is contract testing. Let’s look at how it helps us test microservices.

Monolith vs Microservices

A modular monolith runs in a single process, which can be either a strength or a weakness depending on the context. Its modules talk to each other through local function calls, so delivering business functionality does not involve the network. As a result, the unit and integration tests that verify those modules are written against their public API. Modifying a module’s API (for instance, changing a method’s signature) leads to a compilation error. That makes it difficult to break the communication contract between modules inside a monolith.

With microservices, the situation is more complex. To deliver business functionality, they have to talk over the network, and the compiler can no longer help us. The modules are independent and distributed – in practice, two separate applications, quite likely with their own testing and deployment pipelines.

How to ensure that services get along before we go to production?

Integration testing

Integration testing is probably what most people think of first. It is a fairly obvious answer to this question, and a partly correct one. Before going further, though, I’d like to explain what I mean by integration tests, since people define them in different ways. For me, an integration test checks whether separately developed parts of the system work together correctly and deliver functionality that meets the requirements. Such a test may verify

  • an exposed API (e.g. REST API)
  • interaction with another service
  • message production and consumption
  • the integration with the database

Integration tests should be executed in isolation. In other words, no services need to be running for the tests to work. In fact, we should be able to run them without any network connection at all.

Integration tests are well suited to checking whether a service (or module) works properly with other parts of the system, such as the database or other services. Their primary purpose should be confirming that the business functionality behaves as intended. Sadly, I have often seen them used to verify communication contracts between services. This approach comes with some drawbacks, so let’s take a closer look at them.

communication%20contracts%20between%20services

Suppose we want to verify how two services – order and payment – communicate. The latter exposes a REST API for starting a payment. The order-service is responsible for creating orders, and it calls the payment-service to create a payment for each of them. To confirm that this communication works, we need at least two sets of integration tests:

  • checking that the API meets the requirements. These tests verify that the service returns the expected output for a given request, including the path, headers, and body structure. They have to live in the payment-service and should imitate the order-service calling the API (see Code 1).
  • checking that the order-service calls the payment-service’s API correctly. These tests verify that the HTTP client invokes the payment-service properly. They have to live in the order-service and rely on stubs of the payment-service (see Code 2).

order%20and%20payment

The payment-service’s exposed API was covered by the integration test below, written with Spock and RestAssured.

def "should initiate payment"() {
   given:
   //skipped

   def request =
           """
           {
               "accountId": "$accountId",
               "orderDescription": "My first order",
               "quota": 128.5,
               "paymentType": "TRANSFER_ONLINE",
               "dueDate": "$dueDate" 
           }       
           """

   expect:
   given()
           .contentType(APPLICATION_JSON_VALUE)
           .accept(APPLICATION_JSON_VALUE)
           .and()
           .body(request)
           .when()
           .put("/payment/%s".formatted(paymentId))
           .then()
           .assertThat()
           .statusCode(200)
           .body("status", equalTo("FINISHED"))
           .body("paymentExternalId", equalTo("ext-%s".formatted(paymentId)))
}

Code 1. payment-service API integration test

The test proves that the API behaves the way the payment-service team intended. It verifies that the API accepts the specified request and returns the expected response. It should also run the relevant business logic (those checks are omitted in the test).

order%20payment

Meanwhile, the order-service team, acting as the client, wrote integration tests for the order creation logic. This test is also written in Spock, and it stubs the payment-service with WireMock.

def "should create order and initiate payment"() {
   given:
   //skipped

   def expectedRequest =
       """
       {
           "accountId": "$accountId",
           "quota": $quota,
           "paymentType": "TRANSFER_ONLINE",
           "dueDate": "$dueDate"                       
       }       
       """

def paymentServiceResponse =
       """
       {
           "paymentExternalId": "ext-$orderId",
           "status": "FINISHED"
       }       
       """

and:
wiremockServer.stubFor(put(urlMatching("/payment/([a-fA-F0-9-]*)"))
       .withRequestBody(equalToJson(expectedRequest))
       .willReturn(
               aResponse()
                       .withStatus(200)
                       .withHeader("Content-Type", "application/json")
                       .withHeader("Accept", "application/json")
                       .withBody(paymentServiceResponse)
       )
)

expect:
sut.createOrder(accountId, orderId, quota).paymentProcessed() == true

}

Code 2. Order creation integration test with a stubbed payment-service

Here, stubs of the payment-service are used to verify the order creation logic. The test assumes certain things about how the called service behaves (as captured in the stubs).

So what is the problem with these tests? In fact, they work fine as long as each team knows exactly how the other side behaves. Both need to know what the expected request and the returned response look like – ideally with no need to ever change the contract between them.

As a result, whether the services work together depends on people’s communication and knowledge. Picture a member of the payment-service team who spots inconsistent naming in the response and decides to rename the status field to paymentStatus. Renaming the field and updating the integration test so it passes is easy, and the change can go to production. But unless the client’s code is adjusted too, communication between the services breaks. Once the producer is deployed to production, calls start failing, the service becomes unavailable, and the company loses money.

To tell clients about an upcoming breaking change in the API, the payment-service team needs to know who those clients are. On top of that, they must judge whether the change affects them, which requires knowing how the clients use the API. Integration tests can’t tell them that. They only capture certain assumptions about the contract and offer no guarantee that this is how clients actually use the API in production.

Another drawback of this approach is that the work gets done twice. Testing the communication between two services requires two sets of tests, which typically live separately in the client’s and the producer’s repositories. Even worse, nothing keeps them in sync. When a client team changes the contract, the producer’s code doesn’t reflect it, and the other way round.

Image 1. Integration tests for each side live in separate repositories and are not kept in sync.

Finally, API integration tests tend to be rather tedious and repetitive. The when section calls the API under test, and then the test checks that the expected output came back – field by field, for every field in the response.

Is there a better way to test this? We’ll get to that later in the article.

End-to-end testing

Let me start by defining end-to-end tests. These are tests that check whether the entire system behaves as expected. They run against real service instances, with no isolation, typically deployed to what is called a stage environment. Their job is to confirm that key business scenarios work correctly.

End-to-end tests don’t suffer from the problem described above, but they have drawbacks of their own. One of the most significant is that every microservice has to be deployed and running before the tests can be executed. Setting up and maintaining such an environment is therefore hard. The approach is also fairly costly, because it needs infrastructure and time to run the tests. Checking the contract between two services can’t be done in isolation – we need infrastructure with the code already deployed. That is clumsy and goes against the microservice philosophy of loosely coupled services that can be deployed independently.

Contract Tests to the rescue

Contract tests sit somewhere between integration tests and e2e tests. The idea lets us verify, in isolation, that two services communicate correctly. It rests on a simple assumption: if the consumer and the producer are each tested independently against the same contract, they will be able to communicate properly. That is the core of contract testing. There is exactly one shared contract describing how the services talk to each other, whether synchronously or asynchronously, and both the consumer’s and the producer’s tests are derived from it. Integration testing works differently – there we have two independent sets of tests with no automated way of keeping them in sync.

In practice, the contract is a document that describes the communication using a DSL (Domain Specific Language). Depending on the framework, it can be expressed in different languages and notations, such as Java, Groovy, JSON, YAML, and others. Below is a sample contract written in Java (Pact framework).

builder
      .given("test PUT)
        .uponReceiving(“PUT REQUEST")
        .path(“/payment”)
    .headers(expectedHeaders)
        .method(“PUT”)
      .willRespondWith()
        .status(200)
        .headers(headers)
        .body(“{\”accepted\”: true}")

Producer and consumer?
These two terms come up all the time when discussing contract tests.
A producer (sometimes called a provider) is a service that exposes an API. In message-driven communication, it is the service that publishes messages.

A consumer is a service that uses the API exposed by the producer or consumes the messages it publishes.

Throughout this article, the producer is the service exposing the API, and the consumer is the one using it.

How does contract testing help?

Let’s now look at how it solves the problems of integration testing.

The first problem was that one side could break the contract without telling the other. That was possible with integration tests, but contract tests make it difficult, since there is a single, shared definition of the contract. When the producer (the payment-service team) changes, say, the response, the change has to be made in the contract as well, or the producer’s tests will fail. And once the contract changes, the consumer’s tests are updated too and will likely fail if the change isn’t backward compatible. This pushes the teams to tell each other about changes, with the contract as the basis.

consumer%20producer

Since every detail of the communication is recorded in the contract, the producer team (payment-service) knows precisely who uses its API and in what way. Contracts typically live in the producer’s repository (or in a separate, dedicated one). When several clients use the same API, each integration gets its own contract, because contracts are defined per client. This lets the producer team identify the callers easily. Suppose we want to remove or modify a field in the response. With contracts agreed with every client, we can just check whether that field is used. This makes evolving APIs much more flexible.

Depending on the framework, we may not have to maintain any tests or stubs at all. The producer’s tests are generated from the contract, so we don’t write them ourselves, and they verify that the API behaves as the contract specifies. On the consumer side, tests rely on stubs that are generated from the very same contract. This keeps both sides in sync when it comes to communication and its tests, and it also cuts down the amount of hand-written code.

Lastly, the contract serves as a form of documentation of how the services are coupled. Both sides of the communication create it and agree on it, so it records their arrangements. Some frameworks can even produce OpenAPI documentation from the contract, giving us a single source of truth about the API.

In short, with this approach the only thing to maintain is the contract, which describes how the services communicate. Both producer and consumer tests are run against the rules defined there. This gives us confidence that each side is implemented correctly and that they can work together.

Who owns the contract?

As I explained, contract testing has a single place where the contract is defined. This naturally raises the question of who is responsible for defining and maintaining it. Depending on how the services involved (and their teams) relate to each other, there are two testing approaches:

Consumer-Driven Contract Testing

When the producer is a downstream service that should try to meet the consumers’ needs, the Consumer Driven strategy applies. Here, the consumers shape the communication contract by telling the producer how the API should look. Naturally, the producer team should review, discuss, and confirm such an agreement. This typically happens by the consumer team opening a Pull Request in the contracts repository, which the producer team then reviews and eventually implements.

Consumer-Driven%20Contract%20Testing

This approach has a few benefits:

  • because the contract is written down and approved, the consumer team doesn’t need to wait for the producer to build the feature. They can generate stubs from the contract and use them to test the service’s integration.
  • the producer knows who uses its API and in what way

With this approach, the consumer and producer teams might, for instance, coordinate their work using the following flow:

  1. Suppose the team in charge of order-service (consumer) is building a new feature that needs to integrate with payment-service (producer). While practicing TDD, they find they need a stub of the API.
  2. The teams talk through the contract and define it with a DSL. It is created in, and committed to, the payment-service repository.
  3. Tests for the producer’s API are generated from the contract. At this point they fail, because the implementation is missing.
  4. The contracts and tests are pushed to the repository. A pull request may also be opened for the producer team to pick up.
  5. Stubs of payment-service are generated alongside the tests and may be published to an artifact repository (e.g. Artifactory).
  6. Using the stubs, the consumer team can complete its tests.

This demonstrates that the producer doesn’t need to finish its implementation for the consumer to move forward with its own. Naturally, nothing can go to production until the producer team implements it, but the consumer isn’t held up.

In the end, the producer team must implement the API in line with the contract. Once all tests pass, the service can be deployed to production. Since the consumer’s HTTP client and the producer’s API were both built and tested against one contract, we can be confident the services will communicate correctly in production.

There is one challenge, though. All of this only works if every consumer plays by the rules. A single consumer that doesn’t create a contract can undermine the whole idea. This is usually easy to enforce by requiring consumers to create and agree on a contract before they can integrate with our service. It can also be enforced technically, for example by blocking ingress to our service.

Producer-Driven Contract Testing

Conversely, our service might be used by many parties, most of which we may not even know. In that situation, we can’t tell exactly who wants to use our API or how. This approach also fits when we don’t want clients (consumers) to have influence over our API. Here, the producer is an upstream service: it defines the API contract and tells consumers how to integrate with it. Consumers can treat these contracts as documentation, and they can also use them to test their integration with the producer without deploying code to a server.

producer%20concumers

Contract testing’s trap

I have often seen people overuse contract testing, and I made the same mistake myself. By overuse I mean relying on contract tests instead of unit or integration tests. Contract testing is meant for verifying the contract of services that communicate synchronously or asynchronously, not for validating logic.

As an example, take the payment-service’s API, which returns a status field in its response. Depending on the scenario, its value can be FINISHED, IN_PROGRESS, or FAILED. Yet which value is returned is internal logic of the business process. That should most likely be covered by unit tests, not contract tests. There is no need for a separate contract test scenario for each value of the field – a single one that checks the field is returned with any of those values is enough.

The situation is different when a particular request produces a different response. It could be, for instance, a flag that causes an additional field to appear in the response, or an error case in which the producer responds with a 400 HTTP status to an invalid request. Such cases should be covered by contract tests, because they represent a distinct communication situation.

Taking it further, if a 400 code is returned in 3 scenarios and the responses differ only in the message, a single contract test scenario is enough.

Summary

We began by highlighting the difficulties of testing communication in a microservice architecture and looked at two popular ways of testing it – integration and e2e testing. We then introduced contract testing as a way to address those problems.

Contract testing is an approach that makes it possible to test communication (synchronous and asynchronous alike) between services in isolation. Its central idea is the contract – the place where both sides agree on and record the rules of communication. The contract is defined in one single place, so the parties don’t need to coordinate whenever it changes. Tests for both the producer and the consumer are built on this contract, which guarantees that if they pass, the services will work together in production.

After that, I described how the Consumer Driven and Producer Driven approaches differ and when each one fits. Finally, I shared a tip on avoiding the trap of overusing contract testing.

In upcoming articles, I will present the two most widely used contract testing frameworks – Spring Cloud Contract and Pact. Stay tuned!