Contract tests for an event-driven application built on Kafka and Spring Cloud Contract
This article shows how Spring Cloud Contract can be used to verify asynchronous communication between microservices that rely on Kafka as the message broker.
Earlier, I demonstrated how to write contract tests for synchronous communication carried out over a REST API. That is a fairly simple and popular case, and most contract-testing articles you can find online use it. It is much harder to find guidance on testing asynchronous communication built on top of a message broker.
Here, I will walk through testing asynchronous, event-driven communication that uses Kafka as the message broker. You will need a basic grasp of Kafka to follow along, because describing how it works goes beyond what this post covers. So, if Kafka is new to you, I recommend getting familiar with its core ideas first.
If you are interested in how Kafka and event-driven architecture support the development of distributed systems, I recommend the article below.
The earlier parts of this series are built around an order in an e-commerce platform that gets paid automatically. Two services, order-service and payment-service, talk to each other through a REST API – order-service invokes the API of payment-service. In this part, I reversed the dependencies to show how the same use case can be implemented with an event-driven approach. Now order-service announces that an order has been created by emitting an event, which lands on a Kafka topic. payment-service is one of the subscribers of that topic and starts a payment for each order.
In contract-testing terms, this change makes order-service the producer and payment-service the consumer. To keep things simple, the events are serialized as JSON. Each one carries metadata (eventId and the time it occurred) along with details of the created order.
{
"eventId": "9c465a8f-bf77-4f3a-b357-d760ced8b26f",
"occuredAt": "2023-12-02T13:00:00",
"orderId": "59404950-bc8d-4722-a521-0dee64b680d6",
"accountId": "5c2c2407-7202-4720-9a45-f0d0f8175ef1",
"total": 120.8
}
This tutorial explains how such communication can be tested with Spring Cloud Contract. Just as with REST API testing, the first step is to define the contract.
Defining the contract
The contract can be expressed with the DSL that Spring Cloud Contract offers, written in Java, Kotlin, Groovy, or Yaml. Below is the Groovy contract for the OrderCreated event emitted by order-service:
Contract.make {
description("Should produce OrderCreated event when order was created")
input {
triggeredBy("createOrder()")
}
label("triggerOrderCreatedEvent")
outputMessage {
sentTo("OrderCreated")
body([
eventId: anyUuid(),
occurredAt: anyDateTime(),
orderId: anyUuid(),
accountId: anyUuid(),
total: $(consumer(regex("(\\d+\\.\\d+)")))
])
}
}
As you can see, it differs a bit from a REST API contract. That is to be expected, since this kind of communication follows different rules. No server handles requests from the consumer; instead, the producer (here, order-service) emits an event as the outcome of a business operation. Because of that, the test has to trigger it somehow. The input section specifies what causes the message to be sent – in this case, the createOrder() method. The generated tests make use of it, as you will see later.
On the consumer side, we also need a way to trigger delivery of the message to the Kafka consumer. The label method takes care of that. I named the label triggerOrderCreatedEvent, and you will need to know it when writing the consumer’s tests.
Lastly, the outputMessage section describes where the message goes and what it contains. The sentTo() method states that the message is sent to the OrderCreated topic. The body() method declares that the message body should hold 5 fields of the specified types. Just like in the article on REST API testing, we do not put concrete field values here. Rather, we state that eventId, orderId, and accountId must hold any valid UUID, occurredAt (when the event happened) must be a date-time value, and total must be a positive double.
The contract belongs in /src/contractTest/resources, and subdirectories can be used to group contracts by consumer. Here, order-service is the producer, so its contracts live in this project under the directory
order-service/src/contractTest/resources/contracts/paymentService.
Testing the producer
Generating tests and stubs from the contract and running the producer’s tests requires the Spring Cloud Contract dependency and plugin.
plugins {
id 'org.springframework.cloud.contract' version '4.0.0'
}
dependencies {
testImplementation 'org.springframework.cloud:spring-cloud-starter-contract-verifier:4.0.0'
}
For brevity, I left out the other dependencies and plugins needed to run the tests, such as Spring Boot Test or Spring Kafka. In Gradle’s build.gradle file, I also added the minimum configuration the framework needs:
contracts {
baseClassForTests = "pl.rmaciak.payment.BaseContractTestsSpec"
testFramework = "SPOCK"
}
This states that the generated tests will be written in Spock and, most importantly, that they will extend the BaseContractTestsSpec base class. That class deserves a closer look.
When testing a REST API, the base class starts the server (or a server mock) that the tests run against. There, the server is the producer exposing the REST API, and the tests check whether that API satisfies the requirements set out in the contracts. Event-driven cases work differently. The producer exposes no API; it publishes a message to the Kafka broker instead. The tests have to check whether that message satisfies the requirements stated in the contracts. So we need to intercept the message sent to the broker and verify it against the contracts.
There are several ways to intercept the message. One option is to run the broker in a Docker container using Testcontainers with Kafka’s image. The contract test fires the functionality that emits the event, and the event goes to the matching topic on the freshly started, short-lived Kafka broker. That is essentially what the BaseContractTestsSpec class is responsible for.
@SpringBootTest(webEnvironment = NONE, classes = [OrderApplication.class])
@AutoConfigureMessageVerifier
@Testcontainers
abstract class BaseContractTestsSpec extends Specification {
@Shared
static KafkaContainer kafka = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:latest"))
@Autowired
private OrderCreator orderCreator
@DynamicPropertySource
static void kafkaProperties(DynamicPropertyRegistry registry) {
kafka.start()
registry.add("spring.kafka.bootstrap-servers", () -> kafka.getBootstrapServers())
}
void createOrder() {
orderCreator.createOrder(
UUID.randomUUID(),
UUID.randomUUID(),
BigDecimal.valueOf(120.8)
)
}
}
The class declares a Kafka container based on the latest image of Confluent’s community edition of Kafka (the Kafka field). The container is started in the method marked with @DynamicPropertySource, which also points the spring.kafka.bootstrap-server property at the address the container exposes. Thanks to this, Spring can publish and consume events through the containerized broker. The createOrder() method is the one named as the triggering input in the contract. All it does is run the order creation logic via the injected OrderCreator, which results in the event being published.
With BaseContractTestsSpec implemented this way, the service logic merely publishes messages to the Kafka instance running in the container. What we still lack is a way to receive the message and check it against the contract. A Message Verifier handles that. Notice the @AutoConfigureMessageVerifier annotation on the class – it sets up and configures the beans Spring Cloud Contract uses to verify messages. The framework ships with a number of ready-made verifiers for JMS, Apache Camel, Spring Integration, and more. Sadly, none of them targets Kafka brokers, so we need to write one ourselves.
Spring Cloud Contracts offers the MessageVerifierReceiver interface, whose implementation lets the test receive messages from Kafka. It declares two overloaded receive() methods, both returning the Message interface from Spring Messaging. Spring Cloud Contract validates that message against the contract. To fetch the message from the broker, we can write a test consumer – in this example it stores the messages in a set. It listens only to the OrderCreated topic, so an event sent to the wrong destination will not be picked up and the test will fail. The receive() methods take a message from that set and return it if one is present. Naturally, consuming messages can take a while, so the check is repeated every few seconds.
class KafkaEventVerifier implements MessageVerifierReceiver<Message<?>> {
private final Set<Message> consumedEvents = Collections.synchronizedSet(new HashSet<Message>())
@KafkaListener(topics = ["OrderCreated"], groupId = "order-consumer")
void consumeOrderCreated(ConsumerRecord payload) {
consumedEvents.add(MessageBuilder.createMessage(payload.value(), new MessageHeaders(emptyMap())))
}
@Override
Message receive(String destination, long timeout, TimeUnit timeUnit, @Nullable YamlContract contract) {
for (int i = 0; i < timeout; i++) {
Message msg = consumedEvents.stream().findFirst().orElse(null)
if (msg != null) {
return msg
}
timeUnit.sleep(1)
}
return consumedEvents.stream().findFirst().orElse(null)
}
@Override
Message receive(String destination, YamlContract contract) {
return receive(destination, 5, SECONDS, contract)
}
}
The architecture of this approach looks like this:
At this point, we have all we need to validate the message. It gets published to Kafka and picked up by our verifier. Spring Cloud Contract takes the consumed message and checks whether it matches what the contracts expect. This happens in the generated tests, which the framework produces when the application is built. You can generate them with the following command:
./gradlew check
Here is the test generated from our contract:
@SuppressWarnings("rawtypes")
class PaymentServiceSpec extends BaseContractTestsSpec {
@Inject ContractVerifierMessaging contractVerifierMessaging
@Inject ContractVerifierObjectMapper contractVerifierObjectMapper
def validate_shouldProduceOrderCreatedEvent() throws Exception {
when:
createOrder()
then:
ContractVerifierMessage response = contractVerifierMessaging.receive("OrderCreated",
contract(this, "shouldProduceOrderCreatedEvent.yml"))
response != null
and:
DocumentContext parsedJson = JsonPath.parse(contractVerifierObjectMapper.writeValueAsString(response.getPayload()))
assertThatJson(parsedJson).field("['eventId']").matches("[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}")
assertThatJson(parsedJson).field("['occurredAt']").matches("([0-9]{4})-(1[0-2]|0[1-9])-(3[01]|0[1-9]|[12][0-9])T(2[0-3]|[01][0-9]):([0-5][0-9]):([0-5][0-9])")
assertThatJson(parsedJson).field("['orderId']").matches("[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}")
assertThatJson(parsedJson).field("['accountId']").matches("[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}")
assertThatJson(parsedJson).field("['total']").matches("(\\d+\\.\\d+)")
}
}
The test class extends our BaseContractTestsSpec, and its when section just invokes the createOrder() trigger method. As a result, the message is published to the Kafka broker and then fetched by our consumer, so the receive() call in the then section eventually returns it. In the end, each field of the message is asserted. The test sits in the build/generated-test-sources directory.
This stage of testing checks whether a valid message reached the right topic on the Kafka broker. When the tests pass, we can be confident the message satisfies what the contract expects. You do have to write some code, mostly because Kafka lacks a built-in verifier. Still, that code is fairly generic and can easily be extended to cover other events as well.
Testing the consumer
In REST API contract tests, building the application produced Wiremock stubs reflecting the contract’s assumptions, and the consumer’s tests could use them. Event-based communication through a message broker involves no direct HTTP connection between producer and consumer. It depends on messages that are published to and consumed from a broker sitting between the services. Testing the consumer therefore really means checking whether it can understand and handle the event emitted by the producer.
Since the consumer consumes events, it typically implements an event listener. In the test, we want to confirm that a message following the contract’s rules is properly consumed and processed. To do that, we have to send a sample message to Kafka for the consumer to pick up and process. In our case, payment-service is the consumer, and it consumes events coming from order-service.
To create and publish a message matching the contract, we have to know its shape. That is why the JAR built for the producer contains no Wiremock stubs but the contract itself – exactly the one we defined on the producer side. It tells us:
- where the event goes (topic)
- how the message is structured
- what causes the message to be sent
The contract is packaged in a JAR file located in the build/libs directory (order-service-0.0.1-SNAPSHOT-stubs.jar). The JAR can be pushed, for example, to the local Maven repository (.m2 directory). With the contract in hand, we can write tests that verify the logic of payment-service. As a reminder, this service should start a payment whenever an OrderCreated event arrives.
Testing this also requires a Kafka broker, and once again we run Kafka as a Docker container with Testcontainers. Sending the sample message to the broker takes two things:
- the message definition (the contract stored in the JAR)
- a way to send the message to the broker. As before, Kafka has no built-in mechanism for this, so we need to write it ourselves.
Here is the test class with a single test scenario:
@SpringBootTest(webEnvironment = NONE, classes = [PaymentApplication.class, TestConfiguration.class])
@AutoConfigureStubRunner(ids = ["com.rmaciak:order-service:0.0.1-SNAPSHOT:stubs:8088"], stubsMode = LOCAL)
@Testcontainers
class EventHandlerSpec extends Specification {
@Shared
static KafkaContainer kafka = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:latest"))
@Autowired
PaymentRepository paymentRepository
@Autowired
StubTrigger stubTrigger
@DynamicPropertySource
static void kafkaProperties(DynamicPropertyRegistry registry) {
kafka.start()
registry.add("spring.kafka.bootstrap-servers", () -> kafka.getBootstrapServers())
}
def "should initiate payment on OrderCreatedEvent"() {
given:
def conditions = new PollingConditions(timeout: 10, initialDelay: 1.5, factor: 1.25)
when:
stubTrigger.trigger("triggerOrderCreatedEvent")
then:
conditions.eventually {
def payments = paymentRepository.getAll()
assert payments.size() == 1
assert payments[0].paymentStatus() == FINISHED
}
}
}
Among other annotations, it carries @AutoConfigureStubRunner, which you may already be familiar with. In our test, it supplies the contract used to generate the event. There is also a Kafka container, declared and started in a method marked with DynamicPropertySource, as described in the producer section. The most interesting field in this test class is stubTrigger – an object from Spring Cloud Contract that lets you trigger sending the message. It is used in the when section of the test method. Here, I trigger by label, and the triggerOrderCreatedEvent label comes from the contract. Spring Cloud Contract offers other triggering mechanisms as well, described in the documentation.
This scenario verifies that once the service consumed the OrderCreated event, a payment was created and carried out (its status is FINISHED). The payment and status check is wrapped in PoolingCondition, since publishing and consuming the event happen asynchronously and will only eventually complete.
The test is pretty straightforward: it triggers the message and checks that the expected logic ran. But what actually sends the message? For that, we implement a dedicated interface, MessageVerifierSender, which sends the message to Kafka.
@Configuration
class TestConfiguration {
@Bean
MessageVerifierSender<Message<?>> standaloneMessageVerifier(KafkaTemplate kafkaTemplate) {
return new MessageVerifierSender<Message<?>>() {
@Override
void send(Message<?> message, String destination, @Nullable YamlContract contract) {
kafkaTemplate.send(message)
}
@Override
<T> void send(T payload, Map<String, Object> headers, String destination, @Nullable YamlContract contract) {
kafkaTemplate.send(
createMessage(payload, new MessageHeaders(Map.of(TOPIC, destination)))
)
}
}
}
}
It just publishes the event with KafkaTemplate from SpringKafka. The payment-service listener then consumes and processes it.
Both the producer and the consumer of the event are now tested against one and the same contract, which guarantees that each side of the exchange follows the rules it declares. Even though the services do not talk to each other directly and a broker sits in between, contract tests let us make sure their communication is correct.
Why do I need contract tests for Kafka messages?
At this point you might ask – why bother with contract tests for Kafka messages? Messages are usually not JSON; they tend to use schema-based formats such as Avro together with Schema Registry, which guarantee message compatibility. There are two answers:
- For whatever reason, you do not want to, or cannot, use something like Avro + Schema Registry
- A schema only checks the semantic correctness of a message. It does not guarantee that the message was published to and consumed from the right topic, nor does it check whether the message is valid for a given scenario. For instance, a message can have optional fields. If none of them are present, the message is still semantically valid. Yet in business terms such a message makes no sense – say, it needs at least one optional field to be valid.
So even with a schema-based message format and Schema Registry in place, contract tests are worth adding to make sure the communication is correct not just semantically but also with respect to functional requirements.
Summary
This article demonstrated how Spring Cloud Contract can be used to test asynchronous communication between microservices that use Kafka as the message broker. We began by defining the contract for that communication, and then tested both the producer and the consumer against it. Along the way, I pointed out several differences between how the framework handles REST API testing and event-based communication.
As you may have noticed, testing Kafka events with this framework is not particularly easy (compared with, for example, REST API testing). You need to write some code to verify that the message published to and consumed from the broker is correct. As mentioned, this is mostly because Spring Cloud Contract has no built-in mechanism for testing Kafka messages, so we have to build it ourselves. Even so, I hope this article makes the topic easier to grasp and helps you implement such tests in your own project.
The next and final article in the series will cover testing Kafka contracts with a Pact broker. I encourage you to read it and choose the approach that fits your project best.