Estimate project
← All posts

Contract Tests Using Pact

The third installment of the contract testing series. This time, the focus is on Pact – another tool that helps developers write contract tests. Find out how it works, what advantages it brings, and how it mainly differs from Spring Cloud Contract.

12 min read
JavaMicroservicesTestingContract testsPact

This is the third installment of the series on contract testing. Earlier, I explained what contract testing is and which benefits it brings, and then presented Spring Cloud Contract – one of the most widely used frameworks for such tests, particularly in the JVM ecosystem.

This part covers another popular tool that helps developers write contract tests: the Pact framework. You will find out how it works, what advantages it offers, and how it mainly differs from Spring Cloud Contract. Once you finish this series, you will have solid background knowledge for testing the communication of systems integrated through a REST API.

The examples in this article revolve around communication between the order-service and the payment-service, which happens over a REST API. Tests are written in Groovy with Spock, and the projects are built with Gradle. As a reminder, this is how the services communicate:
communication between services 1
communication between services 2

Defining the contract

With Spring Cloud Contract, testing the communication begins with defining the contract. The Pact framework takes a different route. Rather than writing a contract, we begin by describing what we expect from the interactions between the services. How is that different? The only difference is that what we write is not quite the document shared between the communicating parties – Pact takes these expectations and generates the final contract from them. Let’s look at how to do this with Groovy and Spock.

class OrderCreatorPactSpec extends Specification {
    (...)

    def "should create order and execute online payment"() {
        def accountIdVal = randomUUID()
        def orderIdVal = randomUUID()
        def quotaVal = BigDecimal.valueOf(1040.42)

        given: 'payment-service executes online payment correctly'
        def paymentService = new PactBuilder()
        paymentService {
            serviceConsumer "order-service"
            hasPactWith "payment-service"
            port 8088

            uponReceiving('a request to execute online payment')
            withAttributes(
                    method: 'PUT',
                    path: regexp("/payment/$UUID_REGEXP", "/payment/$orderIdVal"),
                    headers: [
                            "Accept"      : APPLICATION_JSON_VALUE,
                            "Content-Type": APPLICATION_JSON_VALUE
                    ]
            )
            withBody {
                accountId uuid()
                paymentType string("TRANSFER_ONLINE")
                quota decimal()
                dueDate datetime()
            }
            willRespondWith(
                    status: 200,
                    headers: ['Content-Type': APPLICATION_JSON_VALUE],
                    body: {
                        paymentExternalId("ext-$orderIdVal")
                        status('FINISHED')
                    }
            )
        }

        when:
        PactVerificationResult result = paymentService.runTest {
            assert sut.createOrder(accountIdVal, orderIdVal, quotaVal, TRANSFER_ONLINE).paymentProcessed()
        }

        then:
        result == new PactVerificationResult.Ok()
    }
}

The first thing to notice is that these expectations take the form of a test in the order-service, i.e. the consumer. Pact only supports Consumer Driven Contract Testing, so the expectations typically live on the consumer side.

Besides defining what we expect from the interaction, the test above also checks that the consumer fulfills those expectations. Specifically, the test runs the order creation logic, which uses an HTTP client internally. Then, in the ‘then’ section, it verifies that the client satisfies the defined expectations.

In contrast to Spring Cloud Contract¹, Pact was designed from day one as a technology-agnostic tool. That is why it offers numerous APIs and DSLs in many languages – from JVM languages (Java, Groovy, etc.), through Ruby and Rust, up to C++. Let’s go through the expectations in this test, which are written with the Groovy DSL.

The expectations live in the test’s ‘given’ section. The PactBuilder class offers a fluent DSL for describing the aspects of the communication. Here, it begins by specifying the communicating parties and the port, which makes it possible to identify which contracts to verify when the producer’s tests run.

serviceConsumer "order-service"
hasPactWith "payment-service"
port 8088

Request

Three methods describe the request:

  • uponReceiving, which simply marks where the interactions begin and lets us describe what triggers this interaction, e.g. uponReceiving(‘a request to execute a payment’)
  • withAttributes, which specifies the REST API endpoint’s method, path, and required headers
  • withBody, which lists the fields the request body must contain

As with Spring Cloud Contract, we avoid giving exact expected values whenever we can. Instead, we describe them with general patterns the fields have to match.

For example, the request body doesn’t include a concrete accountId – instead, it’s declared as any uuid with uuid().

withBody {
    accountId uuid()
    paymentType string("TRANSFER_ONLINE")
    quota decimal()
    dueDate datetime()
}

The path, in turn, is described with a regular expression:

path: regexp("/payment/$UUID_REGEXP", "/payment/$orderIdVal")

The second argument’s value is needed because the response references it. Thanks to this construct, the producer’s tests use the supplied value rather than a randomly generated uuid.

Response

For the request described above, the order-service expects the producer to return a particular response. Its status should be 200, and it has to include a Content-Type header set to application/json as well as two fields – paymentExternalId and status. The test defines these expectations with the willRespondWith() method from the Groovy DSL. Many other handy DSL methods exist for describing the contract; see the documentation for details.

willRespondWith(
        status: 200,
        headers: ['Content-Type': APPLICATION_JSON_VALUE],
        body: {
            paymentExternalId("ext-$orderIdVal")
            status('FINISHED')
        }
)

Testing the consumer

With the API expectations in place, we can now test the consumer. The test should check that the client complies with the communication requirements. To test consumers with Pact, we need to add this dependency.

testImplementation 'au.com.dius.pact.consumer:groovy:4.4.4'

It contains not just the DSL described above, but also an API for running the test in Spock. The test runs the order creation logic, which relies on an HTTP client under the hood, and checks that the client calls the payment-service REST API correctly.

when:
PactVerificationResult result = paymentService.runTest {
    assert sut.createOrder(accountIdVal, orderIdVal, quotaVal, TRANSFER_ONLINE).paymentProcessed()
}

then:
result == new PactVerificationResult.Ok()

When we build the order-service application, the tests should pass. As a by-product, Pact produces the contract in JSON format. This is the actual contract, generated from the expectations defined in the test, and the JSON is used to test the consumer and, as I’ll show later, the producer as well. You’ll find the file in the build/pact directory; for the expectations described above, it looks like this (the snippet has been trimmed for readability).

{
  "consumer": {
    "name": "order-service"
  },
  "interactions": [
    {
      "description": "a request to execute online payment",
      "request": {
        "body": {
          "accountId": "e2490de5-5bd3-43d5-b7c4-526e33f71304",
          (…)
        },
        "headers": {
          "Accept": "application/json",
          "Content-Type": "application/json"
        },
        "matchingRules": {
          "body": {
            "$.accountId": {
              "combine": "AND",
              "matchers": [
                {
                  "match": "regex",
                  "regex": "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"
                }
              ]
            },
            (…)
           ]
          }
        },
        "method": "PUT",
        "path": "/payment/f4d13515-8ffd-424a-9c71-f0a52e1ffa31"
      },
      "response": {
        "body": {
          "paymentExternalId": "ext-f4d13515-8ffd-424a-9c71-f0a52e1ffa31",
          "status": "FINISHED"
        },
        "headers": {
          "Content-Type": "application/json"
        },
        "status": 200
      }
    }
  ],
  "provider": {
    "name": "payment-service"
  }
}

Testing the producer

The consumer has stated its expectations and generated the contract. Next, we need to make sure the other side of the communication meets them, which means the contract has to reach the producer somehow. How do we achieve that?

There are essentially two ways:

  • expose the contract ourselves – for example, by giving the URL of the contract hosted on a server, or its location in the local filesystem. This is a rather clumsy approach.
  • use a Pact Broker – a tool for storing, managing, and exposing contracts.

My recommendation is the Pact Broker, since it greatly simplifies testing and releasing.

Pact Broker

The Pact Broker can be seen as a central database holding the contracts between services. It offers several features that streamline the process:

  • it lets you test services against the contract, ensuring that services in particular, tested versions can go to production
  • it provides documentation and visualizations of how the services interact
  • it handles contract versioning and conflict management

You can run, configure and manage a Pact Broker instance yourself. There is also a commercial, fully managed, scalable edition of the tool – Pact Flow.

Publishing contracts to Pact Broker

Publishing contracts to the Pact Broker requires a Gradle plugin. The configuration below specifies that the broker runs on localhost, port 9292.

plugins {
      id "au.com.dius.pact" version "4.3.6"
}

(...)

pact {
    publish {
        pactBrokerUrl = 'http://localhost:9292/'
    }
}

With this in place, contracts can be published with the pactPublish task.

./gradlew pactPublish

The contracts between order-service and payment-service are now in Pact and can be viewed in the broker’s GUI.

Clicking the file icon shows the contract’s details. The interactions we specified as expectations in the consumer’s tests appear here as JSON.

Executing the producer’s tests

The producer application needs to be set up so that it integrates properly with the Pact Broker and checks whether it satisfies the contract’s requirements. This service is built with Spring Boot, so the tests have to integrate with it. First, we add the dependency that provides an API for running the producer’s tests.

testImplementation 'au.com.dius.pact.provider:junit5spring:4.2.21'

The test engine needs to know where the Pact Broker is, so we set it in the test application.properties file.

pactbroker:
  host: localhost
  port: 9292

Lastly, we implement a test class that runs the producer’s checks against the contract published to the Pact Broker.

@SpringBootTest(webEnvironment = RANDOM_PORT)
class ContractVerificationSpec extends Specification {

    @LocalServerPort
    private int port

    @Shared
    ProviderInfo serviceProvider
    ProviderVerifier verifier

    def setupSpec() {
        serviceProvider = new ProviderInfo("payment-service")
        serviceProvider.hasPactsFromPactBroker("http://localhost:9292")
    }

    def setup() {
        verifier = new ProviderVerifier()
        serviceProvider.port = port
    }

    def 'verify contract with #consumer'() {
        expect:
        verifyConsumerPact(consumer) instanceof VerificationResult.Ok

        where:
        consumer << serviceProvider.consumers
    }

    private VerificationResult verifyConsumerPact(ConsumerInfo consumer) {
        verifier.initialiseReporters(serviceProvider)
        def testResult = verifier.runVerificationForConsumer([:], serviceProvider, consumer)

        if (testResult instanceof VerificationResult.Failed) {
            verifier.displayFailures([testResult])
        }

        testResult
    }
}

It might seem rather complex, so let’s walk through what happens step by step.

The ‘@SpringBootTest’ annotation on the class makes the server start with the test configuration on a random port.

Next, we declare two objects needed to run contract verification:

  • ProviderInfo – describes the configuration of the service under test (payment-service), such as its name, host, port, request path, and so on.
  • ProviderVerifier – checks the producer against the defined consumers.

In Spock’s setup methods, we configured the provider with the name payment-service and the port used by SpringBoot. We also specified that the contracts live in the broker reachable at localhost:9292. As a result, the consumers that have a contract with this producer are fetched from the broker along with their expected interactions. This means we don’t need to update the test when, say, an interaction changes or a new consumer wants to integrate with payment-service. Each time the payment-service’s tests run, every contract from every consumer is fetched from Pact and tested.

If you now build this application, the test should pass. But if the API doesn’t match the expectations in the contract, the test fails and reports what went wrong.

This is just one of the ways to run contract tests on the producer side. I picked it because every example in this series uses Spock. Another recommended option, however, is a Gradle plugin for verifying the producer, which cuts down the code needed to test the producer. That kind of testing, though, is beyond the scope of this article.

When using Pact together with the Pact Broker, it’s good practice to publish test results to the broker. This shows whether a particular version of the service meets the consumers’ expectations. To enable it, we need to set two system properties:

  • pact.verifier.publishResults – when set to true, results are published to the broker
  • pact.provider.version – the service version to publish
test {
    useJUnitPlatform()
    systemProperty 'pact.provider.version', project.version
    systemProperty 'pact.verifier.publishResults', 'true'
}

With this configuration, the results are sent to the broker once the tests finish. The main page of the broker’s GUI then shows that the contract between the services was validated recently and that the tests passed.

Pact Broker maintains and displays a version matrix for every integration. You can open it by clicking the grid icon in the list of contracts.

It presents the history of contract validation between the services. We can see that version 1.1.0 contains a bug, so it shouldn’t be deployed. But how does the CD pipeline know whether a given version was tested and may go to production? That’s where the can-i-deploy tool comes in. With this CLI, we can ‘ask’ the Pact Broker whether the service is compatible with, and complies with, the published contracts. For instance, we can check if the latest payment-service version can be deployed to production safely.

pact-broker can-i-deploy --pacticipant payment-service --latest

‘can-i-deploy’ tells us that this version is OK.

If, however, we ask about version 1.1.0, which the matrix above shows as incompatible, a failure gets reported.

pact-broker can-i-deploy --pacticipant payment-service --version 1.1.0

This tool can be plugged into our CD pipeline to check service compatibility before anything goes to production, which guards us against deploying incompatible service versions.

Summary

This article explained how the Pact framework works. We wrote consumer tests confirming that the order service’s code complies with the communication contracts, and I showed how the contract is generated in Pact’s JSON format so it can be shared with the producer.

I also presented the Pact broker as the recommended way to share contracts between the parties. It makes it possible to run the producer’s tests against the contracts defined by every consumer, without the producer’s tests even knowing who those consumers are.

Lastly, I demonstrated how the Pact Broker and the ‘can-i-deploy’ tool can be used in a CD pipeline to keep incompatible services from being deployed to production.

Along the way, I pointed out the key differences between Spring Cloud Contract and the Pact framework. The most important of them are:

  • Pact supports only the consumer-driven approach, whereas Spring Cloud Contract also supports a producer-driven one.
  • Pact’s contracts are language agnostic (JSON) and are generated from the consumer’s tests. Spring Cloud Contract requires you to write the contract manually in one of its DSLs.
  • In Spring Cloud Contract, contracts are kept in, e.g., a Git repository and shared between the parties whenever they need to be, say, updated. Pact comes with a broker that makes sharing and managing contracts easier.
  • Spring Cloud Contract needs a stub of the producer to implement the consumer’s test. That stub is built from the contracts, which typically live in the producer’s repository. With Pact, we begin by writing the consumer’s tests, and they can be completed without any external stub (the integration expectations are part of the tests).
  • Pact offers extra features that make deployment management easier, e.g. can-i-deploy.

I hope you now have a basic grasp of contract testing concepts, along with some knowledge of these two frameworks. That should be enough to start writing contract tests for the REST APIs in your project, with either Spring Cloud Contract or Pact.

A follow-up article will cover how to test event-driven communication with Kafka brokers.

¹Spring Cloud Contract now supports non-Spring and non-JVM languages as well.