Estimate project
← All posts

Contract Testing in Practice with Spring Cloud Contract

A walkthrough of describing a REST API contract with Spring Cloud Contract, covering the key building blocks of the framework and how to set it up.

13 min read
JavaMicroservicesspringTestingContract tests

Contract Testing is a technique for verifying how services talk to each other, with each service checked in isolation. It removes the issues that show up when communication is tested through integration or e2e tests. This article presents Spring Cloud Contract, one of the popular libraries for contract testing.

Being familiar with the general idea of Contract Testing will make it easier to follow the concepts behind Spring Cloud Contract.

Every example here revolves around communication between order-service and payment-service over a REST API. The tests use Groovy with Spock, and both projects are built with Gradle. As a reminder, this is how the two services interact:
image3
image4

Defining the contract

Contracts are the central element of this kind of testing. Spring Cloud Contract lets us describe them with a handy DSL (Domain Specific Language), which can be written in Groovy, Java, Kotlin, or even YAML. A contract specifies how the producer and its consumers communicate. The contract below describes how order-service uses the REST API of payment-service. It is written in Groovy and covers the happy path of an online payment.

Contract.make {
   name "should execute online payment"
   request {
       method 'PUT'
       urlPath $(consumer(regex('/payment/' + uuid())))
       body([
               accountId  : $(consumer(anyUuid())),
            quota      : $(consumer(regex('^(\\d+\\.\\d+)$')), producer(1024.50)),
               paymentType: $(consumer(regex('^TRANSFER_(OFFLINE|ONLINE)$')), producer('TRANSFER_ONLINE')),
               dueDate    : $(consumer(anyDateTime()))
       ])
       headers {
           accept 'application/json'
           contentType 'application/json'
       }
   }
   response {
       status OK()
       body([
               paymentExternalId: "ext-${fromRequest().path(1)}",
               status           : "FINISHED",
       ])
       headers {
           contentType('application/json')
       }
   }
}

Contracts belong in /src/contractTest/resources, where they can be arranged into directories and subdirectories – for instance, one per consumer. Here the contract lives in payment-service/src/contractTest/resources/contracts/orderService, since it concerns order-service.

Request

The key parts of the contract are the request and response blocks. The request block specifies what a call to the API should look like. It declares a PUT method and a path given as a regular expression: /payment/ followed by a UUID. This syntax can seem a bit cryptic at first, and I will come back to it shortly.

Next comes the expected request body, with 4 mandatory fields: accountId, quota, paymentType, and dueDate. Their values may look complex, but we will break them down in a moment. The final part of the request block lists the required headers: the API expects Accept and Content-Type, each set to application/json.

With the basics covered, let’s unpack the parts we skipped, starting with the URL definition.

    urlPath $(consumer(regex('/payment/' + uuid())))

The $(consumer(…)) expression is how we declare so-called dynamic properties – values we prefer not to hardcode in the contract, such as an ID or a timestamp. Hardcoding them would force the consumer to use those exact values in its tests, which is far from ideal. We want the consumer team to have freedom in how it writes its tests, so a regular expression is the better choice. Here we relied on the built-in uuid() method, which provides a regular expression matching a UUID.

What we wrote above is in fact shorthand for the following expression:

urlPath $(consumer(regex('/payment/' + uuid())), producer('/payment/42cc610b-575d-4a52-9fba-4a43e8d3cfe9'))

consumer() specifies which values are accepted when this API is called, and these are the values that end up in the stub. Here, any URL consisting of /payment/ and a valid UUID is accepted. Put differently, the consumer’s tests may use any UUID rather than one particular value.

producer() specifies the value used when the producer’s API is tested. A regular expression won’t do here, since the test must call the API with a concrete URL (e.g. /payment/42cc610b-575d-4a52-9fba-4a43e8d3cfe9). Because the full expression is rather long, Spring Cloud Contract lets us omit the producer() part, as our contract does. In that case, the value used in the API test is generated from the regular expression given in consumer().

The accountId and dueDate fields in the request body are defined the same way. For quota and paymentType, we supplied our own regular expressions.

quota      : $(consumer(regex('^(\\d+\\.\\d+)$')), producer(1024.50)),

paymentType: $(consumer(regex('^TRANSFER_(OFFLINE|ONLINE)$')), producer('TRANSFER_ONLINE'))

Quota must be a positive double, and sadly Spring Cloud Contract offers no built-in method for that, unlike for UUID. The complete list of built-in regular expression methods is in the documentation. Likewise, paymentType is an enum whose value is either TRANSFER_ONLINE or TRANSFER_OFFLINE. So we need to write the regular expressions ourselves and supply the value for the API test (producer()).

Response

The response block describes what the producer sends back for the given request. Most of it is self-explanatory: HTTP status, response body, and headers. The one thing that may come as a surprise is the definition of paymentExternalId.

paymentExternalId: "ext-${fromRequest().path(1)}"

We want this field to always be ext- followed by paymentId (which payment-service receives as a URL path param). Since paymentId is defined by a regular expression (a valid UUID), it can’t be hardcoded – it has to be exactly the value sent in the URL. That’s where the fromRequest() method comes in: it lets the response refer to values from the request. We needed paymentId, the second segment of the path, and the path(n) method gives us access to it. N is the index of the path element, counting from 0. Similar methods exist for reading values from the request body, headers, and so on – they are listed here.

Testing the producer

With the contract in place, we want to check whether the producer’s API actually behaves as described. We don’t need to write any tests by hand for that – they are generated from the contracts. To get this working with Spring Cloud Contract, add the following plugin and dependency to the producer’s project.

plugins {
    (...)
    id 'org.springframework.cloud.contract' version '4.0.0'
}

(...)

dependencies {
    (...)
    testImplementation 'org.springframework.cloud:spring-cloud-starter-contract-verifier:4.0.0'
}

The plugin is what generates tests from the contracts. The snippet above assumes your project already has the dependencies needed to run and test a Spring Boot application (e.g. spring-boot-starter-web and spring-boot-starter-test).

Generating and running the producer’s tests requires a bit of Spring Cloud Contract configuration.

contracts {
    baseClassForTests = "pl.rmaciak.payment.BaseContractTestsSpec"
    testFramework = "SPOCK"
}

At a minimum, only baseClassForTests has to be set. The second parameter says the tests should be written in Spock. baseClassForTests names the class that every generated test extends, and that class must prepare the server used during the tests. There are two ways to do this:

  • mock the server with e.g. RestAssured or MockMvc. This is the default and the recommended approach.
  • launch the server, for example with @SpringBootTest. The tests then hit a real server rather than a mock. To test this way, set testMode in the configuration to EXPLICIT.

Our example uses RestAssuredMockMvc. The remaining configuration options are described here.

Which option should I choose?

It might seem that testing against a real server listening for requests beats mocking it. Keep in mind, though, that contract testing is about communication, so the full production code doesn’t need to be running.

A mocked server that exposes the controllers, with their dependencies stubbed, is usually sufficient. That layer is what defines how communication works. The business logic invoked by the controller isn’t needed and can be stubbed.

To generate the tests and run them, execute:

./gradlew check

When the API matches the contract, the build should succeed. Otherwise, you will get an error describing the problem.

This confirms that the tests were indeed generated and run. Let’s now peek under the hood to see what happened.

Inside the build directory, under generated-test-sources, you will find the OrderServiceSpec class, generated automatically by Spring Cloud Contract.

The class extends BaseContractTestsSpec, which we set in the configuration, and contains a Spock test (also set in the configuration) verifying that the server behaves as the contract specifies. By default, the generated test exercises the server through Spring MockMvc, but the API can also be tested in the following ways:

  • SpringMockMvc from Rest assured. This is the default.
  • SpringWebTestClient from Rest Assured. Requires the testMode parameter to be set to WebTestClient
  • JaxRSClient. Requires the testMode parameter to be set to JaxRsClient
  • There is also the special mode mentioned earlier – EXPLICIT, in which the server is started and real HTTP calls are made.

Below is the test generated for our contract. It builds the request according to the contract, calls the API, and checks that the response matches what the contract expects. Nothing remarkable – apart from the fact that all of this boilerplate was generated for us, so we didn’t have to write it. And that’s just a single scenario. Imagine the others: particular communication cases, error cases, and so on, multiplied by every endpoint. That’s a good deal of work taken off our hands, isn’t it?

def validate_should_execute_online_payment() throws Exception {
    given:
        MockMvcRequestSpecification request = given()
                .header("Accept", "application/json")
                .header("Content-Type", "application/json")
                .body('''{"accountId":"f4459572-b7a8-4d2e-9f49-75009f9e7f6f","quota":1024.50,"paymentType":"TRANSFER_ONLINE","dueDate":"2018-08-18T12:23:34"}''')

    when:
        ResponseOptions response = given().spec(request)
                .put("/payment/5ecbe91f-a1f7-baff-a9b3-55d48cbbf2fb")

    then:
        response.statusCode() == 200
        response.header("Content-Type") ==~ java.util.regex.Pattern.compile('application/json.*')

    and:
        DocumentContext parsedJson = JsonPath.parse(response.body.asString())
        assertThatJson(parsedJson).field("['paymentExternalId']").matches("ext-5ecbe91f-a1f7-baff-a9b3-55d48cbbf2fb")
        assertThatJson(parsedJson).field("['status']").isEqualTo("FAILED")
}

Such tests (typically covering more scenarios) verify that the API is correct. The next step is to check that the consumer calls this API properly.

Testing the consumer

The team behind the consumer service must write tests confirming that their service integrates correctly with the API. These tests can operate at various levels: they may cover just an HTTP client, or they may check that the service’s logic behaves properly when interacting with the other service – which is essentially what integration tests do. The key difference is that with Spring Cloud Contract we don’t write the stubs ourselves. We use the ones generated from the producer’s contract instead.

Looking at the build directory of payment-service once more, you will spot one more JAR file - payment-service-0.0.0.1-SNAPSHOT-stubs.jar

This JAR holds the stubs and can be used to run a WireMock server in the consumer’s tests. Spring Cloud Contract Stub Runner takes care of that.

testImplementation 'org.springframework.cloud:spring-cloud-starter-contract-stub-runner:4.0.0'

It provides the AutoConfigureStubRunner annotation, which accepts several arguments. In the simplest setup, only the ids parameter is required. It specifies which stub JARs Wiremock should use to mock the server.

@SpringBootTest(classes = [OrderApplication])
@AutoConfigureStubRunner(
        ids = ["com.rmaciak:payment-service:0.0.1-SNAPSHOT:stubs:8088"],
        stubsMode = StubRunnerProperties.StubsMode.LOCAL
)
class OrderCreatorWithContractSpec extends Specification {
    (...)
}

Here I pointed to the JAR built from the contracts in payment-service. The string inside the array may look odd at first, so let’s break it down.

  • the first segment is the groupId
  • payment-service is the artifactId
  • 0.0.0.1-SNAPSHOT – the version of the stubs JAR. It can be ‘+’, meaning the latest version is always used
  • stubs is the JAR classifier. It’s the default value and can be omitted.
  • 8088 is the port the WireMock server listens on. If not given, a random one is chosen.

The other AutoConfigureStubRunner parameter used in the example is stubsMode.

It accepts these values:

  • LOCAL – stubs are fetched from the local repository (e.g. the /.m2 directory of Maven), which should be set up by your build tool. Here I used Maven.
  • REMOTE – stubs are fetched from a remote repository. In this mode, the remote repository has to be specified with the repositoryRoot parameter.
  • CLASSPATH – the classpath is scanned for the defined stubs. This is the default.

The annotation’s other parameters are listed in the documentation.

Once the stubs are running, we can test code that makes an HTTP call to the other service.

def "should create order and initiate offline payment"() {
    given:
    def accountId = randomUUID()
    def orderId = randomUUID()
    def quota = BigDecimal.valueOf(120.99)

    expect:
    sut.createOrder(accountId, orderId, quota, TRANSFER_OFFLINE).paymentProcessed()
}

Note that no other stub is set up here – Stub Runner handles all of it.

At this point, both the producer and the consumer are tested against the agreed contract. When both services’ tests pass in the CI/CD pipeline, we can be confident they are integrated correctly and safely ship to production.

Naturally, this article shows just one contract as an example. Doing it properly would likely require more contracts – for instance, when there are particular responses for exceptional cases such as errors. Still, keep in mind the trap that contract testing can lead you into.

Next, let’s look at how Spring Cloud Contract handles both the Consumer Driven and the Producer Driven approaches.

Consumer-Driven Contract Testing with Spring Cloud Contract

As a reminder, in the Consumer Driven Contract approach the consumer, an upstream service, defines the communication. The team owning the producer service (downstream) should then build the API the way the consumer asked for it (while applying common sense). Here is an example of how the consumer team might work.

Spring Cloud Contract supports this workflow. Once the consumer team is happy with the contract, they generate the tests (3) and the stubs (5). They can then take the generated JAR with stubs and include it in their project, which is how the default stubsMode (CLASSPATH) works. A better option, however, is to publish them to a local repository, such as the Maven one (.m2), and use them in the consumer’s tests – that is what LOCAL stubsMode does.

This should only be a stopgap until the producer team builds the API. Once that’s done and every test passes, the stubs JAR should be published to the remote (global) Artifactory. The consumer team can then switch stubsMode to REMOTE, so the stubs are always fetched from the Artifactory. Combine this with setting the stub version to the latest (+ in ids). The consumer’s tests then run against stubs that reflect the contract agreed with the producer, which makes them trustworthy.

Producer Driven Contract Testing with Spring Cloud Contract

Spring Cloud Contract has no dedicated support for Producer Driven Contract Testing. In this approach, the producer owns the contracts, and the consumers simply accept them without any say. So the payment team needs to write the contract and build the API to match it. After that, the stubs can be generated and handed over to the consumers for use in their tests. How they are shared depends on the circumstances – Artifactory, Nexus, a Git repository, and so on. Besides enabling integration testing, the stubs also act as a form of documentation of how the producer’s API behaves.

Summary

This article showed how to describe a REST API contract with Spring Cloud Contract and how to test both the producer and the consumer against it. Along the way, you got to know the core building blocks of the framework and its basic setup. I also covered two ways of using Spring Cloud Contract for contract testing – Consumer Driven and Producer Driven.

Having read this article, you should be ready to introduce Spring Cloud Contract in your own project.