Is testcontainers still necessary with Spring Boot 3.1?
Spring Boot 3.1 has just come out, bringing a completely new spring-boot-docker-compose module. Let's see whether it can take the place of testcontainers in our integration tests and during local development.
Spring Boot 3.0, which came out in November 2022, delivered plenty of exciting new features, and just a few months later the spring team is keeping up the pace – Spring Boot 3.1 has just been released. It brings a huge number of new features, and this article concentrates on the completely new spring-boot-docker-compose module, which can start and stop the services defined in docker-compose for you automatically during local development. As soon as I noticed it, a question popped into my head: “Does this mean I can run my test containers with a good old docker-compose.yml and drop testcontainers altogether?”. Ok… let me clarify what I mean.
Why would I need any of this?
Each of us writes code that relies on external services such as databases or queues to keep some data. Running those external dependencies of our application in docker containers can make the development workflow much simpler. We can run the application against exactly the same versions of external dependencies as in production, which gives us confidence that it works correctly. Starting and stopping them by hand, however, is a bad idea, particularly when we need to run integration tests to check, say, whether the application works correctly with a chosen database.
Luckily, there is an excellent library for this, testcontainers, which can start and stop whatever container we need and takes care of the container lifecycle for us. Thanks to testcontainers, our integration tests run with the application connected to dependencies in the same versions as in production, and everything happens automatically. And there’s more: we can even launch our application locally and let testcontainers run those containers for us. It feels great to type a single command or click a single button in your IDE and see the application up and running! But is the same possible with spring-boot-docker-compose? Could we use it both in tests and in local development, replacing our beloved testcontainers with nothing but spring boot features? Let’s see.
What are we aiming for?
We’ll describe what we want to accomplish with spring-boot-docker-compose using an application built on testcontainers.
The application relies on MongoDB and keeps two simple documents there, which lets us run two simple tests against separate collections; still, we want just one container, and this is one of our goals: run a single instance of a given container for all tests to keep our tests fast.
Here are our MongoDB documents
@Document
public record Article(@Id ArticleId articleId, Author author, Content content) {
}
public record ArticleId(String id) {
}
public record Author(String name) {
}
public record Content(String text) {
}
@Document
public record User(@Id String id, String name) {
}
We’ll create repositories for these documents with the spring-data-mongodb module.
public interface UsersRepository extends MongoRepository<User, String> {
}
Finally, we’ll expose simple HTTP endpoints that return data from these repositories.
@RestController
@RequestMapping("/api/users")
public class UsersController {
private final UsersRepository usersRepository;
public UsersController(UsersRepository usersRepository) {
this.usersRepository = usersRepository;
}
@GetMapping
public Users getAllUsers() {
var users = usersRepository.findAll();
return new Users(users);
}
}
public record Users(List<User> users) {
}
The articles repository and rest controller are nearly identical, so I’ll skip them here.
Setting up integration tests with testcontainers
Next, we’ll prepare the tests and the testcontainers configuration so that our tests run against MongoDB in docker.
Below is a sample test case for the users collection. We store a user in the database and verify that the API returns this user.
@Test
void shouldReturnSavedUser() {
// given
var user = new User(UUID.randomUUID().toString(), "username");
usersRepository.save(user);
// when
Users users = testRestTemplate.getForObject("/api/users", Users.class);
// then
assertThat(users).isEqualTo(new Users(List.of(user)));
}
Practically the same tests are in place for articles.
Now let’s define our container using testcontainers.
@TestConfiguration(proxyBeanMethods = false)
public class TestContainersConfiguration {
@Bean
@ServiceConnection
public MongoDBContainer mongoDBContainer() {
return new MongoDBContainer("mongo:6.0.5");
}
}
To share a single spring context across all integration tests, we typically keep a base class annotated with @SpringBootTest, which our test classes extend; this way only one spring context is created when the whole suite runs, keeping our tests fast. Our containers need the same treatment (they should start just once). All it takes is importing this TestContainersConfiguration into our BaseIntegrationTest
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@Import(TestContainersConfiguration.class)
@ActiveProfiles("test")
public class BaseIntegrationTest {
Psst… In case you missed it, we’re relying on another great Spring Boot 3.1 feature here: Service Connections. You no longer have to define the connection settings for our container (such as host and port) separately. Once a bean is annotated with @ServiceConnection, its connection settings are automatically taken from the running container.
That’s all the configuration required. When we run our tests now, testcontainers brings up a fresh MongoDB container, and our application uses it during the tests.
Container mongo:6.0.5 is starting: 89bd7f517e0eacedae7af96c04cdcf8c8a5878cc8fcec3e57560d1c9b054914c
Container mongo:6.0.5 started in PT1.040507S
Using testcontainers during development
With our tests now running against an automatically started MongoDB, we’ll set up the application in the local environment so that testcontainers starts MongoDB for us there as well, which brings us to the next goal: use one and the same container configuration for running tests and the application in the local env.
For this, we need a new main application inside our test sources, so that it can access the test classpath (testcontainers and the container configuration).
public class TestApplication {
public static void main(String[] args) {
SpringApplication.from(Application::main)
.with(TestContainersConfiguration.class)
.run(args);
}
}
Here we launch SpringApplication with the whole configuration loaded from our main Application, and on top of that we add our MongoDB testcontainers configuration plus a simple class that puts sample data into our database at startup in the local environment.
We also want to seed the database with sample data in the local environment, to show that local and test data don’t affect each other. For this, we’ll use spring profiles and spring conditional auto-configuration so that data initialization is turned on only locally. We have two special profiles here, local and test. This is what our data initializer looks like.
@Configuration
public class InitialDataForDevelopment {
@Bean
@ConditionalOnProperty(name = "data.initialize", havingValue = "true")
public CommandLineRunner commandLineRunner(UsersRepository usersRepository) {
return args -> usersRepository.save(new User(UUID.randomUUID().toString(), "name"));
}
}
As you can see, sample data is initialized only when the data.initialize=true property is set, and thanks to spring profiles we can set data.initialize=true in the local profile alone, so our tests start without any data. That leads to the next important goal: if we launch the main application first, it fills our db with some data, and while it’s still running we want to run our tests in such a way that they don’t see that data – in short, we want the container running in the local env and the tests to be isolated from each other. Testcontainers gives us this by default. When I run my application locally, it starts one container.
Creating container for image: mongo:6.0.5
Container mongo:6.0.5 is starting: cd242f72eb33b89542aeb8710c0a0b9fed12cd4fb0feffb5fa443e2b0199f3e7
Then, with my app running and the initial data in place, I launch my tests, and we can see that testcontainers has started a separate MongoDB container for them.
Container mongo:6.0.5 is starting: 40aa3198e15416f98e029f43b0a7846d17161ad707dfd5ffde6246f6861302bd
So now there are two containers running on my machine, and my tests don’t get in the way of my local development environment.
~/Workspace/demo-spring-boot-testing> docker ps | grep mongo
40aa3198e154 mongo:6.0.5 "docker-entrypoint.s…" 49 seconds ago Up 48 seconds 0.0.0.0:58327->27017/tcp quirky_hawking
cd242f72eb33 mongo:6.0.5 "docker-entrypoint.s…" 29 minutes ago Up 29 minutes 0.0.0.0:57885->27017/tcp cool_jennings
Pretty cool, isn’t it?!
There’s one more important goal that the examples above didn’t demonstrate: run whatever container you need. Our simple example runs just a MongoDB database, but more complex environments require many more services, such as Kafka, Mockserver or Neo4j, and testcontainers supports all of them.
In summary, our goals are:
- Being able to run containers for local development
- Being able to run containers for tests (with just a single instance of a given container)
- Keeping containers for tests and for local development isolated
- Being able to run any container we like
The new spring-boot-docker-compose module in practice
Let’s see whether we can reach our first goal: being able to run containers for local development.
That’s exactly what the module was made for, so it ought to be quite simple. First, we add spring-boot-docker-compose to our dependencies; we use maven, so this goes into our pom.xml file.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-docker-compose</artifactId>
</dependency>
Then we describe our external dependencies in the docker-compose.yml file.
version: '3.9'
services:
mongodb:
restart: always
image: mongo:6.0.5
ports:
- 27017:27017
tmpfs:
- /data/db
Done. When we now start our application, spring boot notices the docker-compose.yml file and launches it for us. Goal number one is reached.
.s.b.d.c.l.DockerComposeLifecycleManager : Found Docker Compose file 'MY_HOME_DIRECTORY_PATH/demo-spring-boot-testing/docker-compose/docker-compose.yml'
o.s.boot.docker.compose.core.DockerCli : Network docker-compose_default Creating
o.s.boot.docker.compose.core.DockerCli : Network docker-compose_default Created
o.s.boot.docker.compose.core.DockerCli : Container docker-compose-mongodb-1 Creating
o.s.boot.docker.compose.core.DockerCli : Container docker-compose-mongodb-1 Created
o.s.boot.docker.compose.core.DockerCli : Container docker-compose-mongodb-1 Starting
o.s.boot.docker.compose.core.DockerCli : Container docker-compose-mongodb-1 Started
o.s.boot.docker.compose.core.DockerCli : Container docker-compose-mongodb-1 Waiting
o.s.boot.docker.compose.core.DockerCli : Container docker-compose-mongodb-1 Healthy
Quite easy, wasn’t it? To be honest, I did have to tweak one setting, since my project is multi-module and spring boot needed to be told where my docker-compose.yml file lives – but that’s just one simple entry in our application.properties. The reason is that it searches for the configuration file in the current working directory, and in a multi-module project the working directory differs depending on whether I run the tests or the main class from my IDE; that’s why I set it by hand, and naturally commands such as ./mnvw test work too. In a single-module project, though, you simply drop the docker-compose.yml file into the project’s root directory and you’re all set.
spring.docker.compose.file=./docker-compose/docker-compose.yml
On to the next goal: being able to run containers for tests (with just a single instance of a given container for all tests)
My first assumption was that no extra work would be needed and it would simply work (the docker-compose.yml file is already there, after all). It turns out, however, that by default spring boot does not start the containers defined in docker-compose.yml during tests – this step is just skipped. A flag controls this behaviour, though, so let’s flip it.
spring.docker.compose.skip.in-tests=false
With this small piece of configuration, our tests can use the MongoDB container from docker compose! Below are logs from a test run showing the container starting. Naturally, every test passed, and the container was started only once for the whole suite.
.s.b.d.c.l.DockerComposeLifecycleManager : Found Docker Compose file 'MY_HOME_DIRECTORY_PATH/Workspace/demo-spring-boot-testing/docker-compose/docker-compose.yml'
o.s.boot.docker.compose.core.DockerCli : Container docker-compose-mongodb-1 Created
o.s.boot.docker.compose.core.DockerCli : Container docker-compose-mongodb-1 Starting
o.s.boot.docker.compose.core.DockerCli : Container docker-compose-mongodb-1 Started
o.s.boot.docker.compose.core.DockerCli : Container docker-compose-mongodb-1 Waiting
o.s.boot.docker.compose.core.DockerCli : Container docker-compose-mongodb-1 Healthy
The second goal can be ticked off.
Goal number three: keeping containers for tests and for local development isolated
This is where I ran into the first downside. Sadly, it isn’t that easy to set things up so that containers in the local environment run separately from those used in tests. The reason is that the new module decides whether to start new containers based merely on a simple check of what the docker compose ps command returns. If any services are already up, no new containers are created, and our tests end up connected to the container started for local development. You can see how this works internally here and here. As a result, trying to run these containers with different databases, docker networks or even docker compose profiles will fail.
The one workaround I came up with is a separate docker-compose.yml file per environment (local and test here), each placed in its own directory. Keeping several docker-compose.yml files with identical configuration isn’t great, but luckily docker compose offers Extensions. To avoid duplication, we can put nearly the entire service configuration into one shared docker-compose.yml, and then reuse that shared definition in docker-compose-local.yml and docker-compose-test.yml with small adjustments – for MongoDB, we have to expose different ports so that two containers can run side by side. These are my new docker-compose files.
./docker/docker-compose-common.yml
version: '3.9'
services:
mongodb:
restart: always
image: mongo:6.0.5
tmpfs:
- /data/db
./docker/local/docker-compose-local.yml
version: '3.9'
services:
mongodb-local:
extends:
file: ../docker-compose-common.yml
service: mongodb
ports:
- 27017:27017
./docker/test/docker-compose-test.yml
version: '3.9'
services:
mongodb-test:
extends:
file: ../docker-compose-common.yml
service: mongodb
ports:
- 27018:27017
I’m aware this isn’t perfect and required some fiddling with docker compose, but all things considered, I’d say this goal is met.
Last but not least: being able to run any container we like
Here we hit a second, though much smaller, issue. At the moment we can’t run just any container out of the box – only a handful are supported (which will likely be enough for most use cases). MongoDB, Redis, PostgreSQL, Cassandra or Elasticsearch, for instance, work with no extra code, but if you need something the module doesn’t cover yet, such as Kafka or Neo4j, you have to write your own DockerComposeConnectionDetailsFactory implementation that builds the ConnectionDetails for you. That’s yet another chunk of code to maintain, so it’s not ideal. Still, it’s most likely only a matter of time before spring-boot-docker-compose gains more DockerComposeConnectionDetailsFactoryies.
Conclusion
As we’ve seen, the new spring-boot-docker-compose module is quite powerful and simple to use. Container isolation may be somewhat tricky to set up, and it forces some duplication across our ‘docker-compose.yml’ files, but it can do everything we did with testcontainers. It also supports fewer containers out of the box than testcontainers does (which shouldn’t matter in most cases, and I’m confident more will arrive in future releases). It seems a big new player has arrived and is going to shake up the status quo.
All in all, I’m afraid I can’t tell you whether you should switch to spring-boot-docker-compose instead of testcontainers. That’s a choice you’ll need to make yourself. I believe this article has covered most of the scenarios in which you’d want docker containers in your development workflow, but bear in mind that the module is quite new and lacks some of the features testcontainers offers, such as starting containers in parallel or reusing containers, to name just two.