Lagom nears its end of life: which migration paths are available
On July 1st, 2024, Lagom reaches end-of-life. What should take its place? How should the migration path be designed? Which options are available?
The opinionated microservices framework Lagom reaches end-of-life on July 1st, 2024. After that date, no more security or bugfix patches will be released. Unsurprisingly, this raises concerns about production applications built on the framework; keeping software current is above all a sound security practice, but it also makes the software easier to maintain, for instance when new features need to be added.
For these and other reasons, software that has reached end-of-life must eventually be replaced. For Lagom, this leads to the question: what should replace it? How should the migration path be designed? Which options are available?
No quick solutions
Lagom is an opinionated framework with no obvious successor, so sadly there are no fast ways to migrate. Whatever route you take, you will have to rewrite at least part of the code (except when forking, see below). Moving off Lagom will therefore cost time and resources. That makes it all the more important to plan the migration, and the resources it will require, carefully.
Fork
Since Lagom is open-source, creating and maintaining a fork remains an option. This is exactly what happened when Akka, which Lagom is built on, switched to a source-available license. The Pekko fork was created and is now being successfully maintained by the community as an Apache project.
For Lagom, though, the community does not appear to have any initiative to do the same. And while a fork lets you keep using the framework without changing your code, maintaining it also costs time and resources.
If you have invested heavily in Lagom, meaning that Lagom underpins your business and many services rely on it, then a fork is certainly a realistic option, at the very least to extend the transition period. If not, it may turn out to be as expensive as the other migration paths, which could bring lower costs in the future.
Survey your systems
Before weighing particular migration options, it is worth reviewing the systems you have. As these are most likely production systems, you probably know a great deal about how they are used, which functionalities work well and which could be improved, much more than during the initial design & implementation phase.
What traffic patterns do the services involved have? Are the systems data-heavy or compute-heavy? Is the load steadily high, or do traffic spikes occur? Which parts of the system see the most use? Should some services be split, or is the current topology perhaps too fine-grained, so that services should be merged?
When evaluating migration options, analysing each system individually is probably the best approach. Quite likely, different services will end up with different migration targets. The fact that a single framework was used to build these services does not mean it is still the right answer; not only do you now have usage data, but the technology landscape has moved on too.
Migration targets: consolidating the tech stack?
Constraints frequently make decision-making easier: when we know we can pick only from a limited subset, there is less room for bikeshedding or flip-flopping over which technology to choose from an almost endless list. Besides, there are usually a few good, comparable technology options!
So think about your migration targets. First, which other technologies does your organization use? A migration like this can be a good opportunity to consolidate your tech stack. That could mean abandoning certain libraries, frameworks or languages entirely, or just standardizing on a particular approach for all new development.
Second, what is your team good at – perhaps they know specific tools and libraries they can put to use when migrating the services. A good developer learns a new technology quite quickly, but when the knowledge is already in place, why not make use of it!
As Lagom provides APIs in Java and Scala, the first decision is whether to stay within the Java / Scala ecosystem. Microservice architectures do permit polyglot deployments, but having too many languages in an organization often causes trouble. Here, too, consolidation may be possible. Moreover, this can be an excellent opportunity to move to newer language versions and stay current in that area as well. Java 21 and Scala 3 both have LTS versions available.
How to start?
It may sound obvious, but whenever you can, begin small. Choose a small, low-impact service as the first one to migrate. You will learn a great deal in the process and be prepared to take on the more complex, higher-impact services. You will also get to know the migration target you chose. Big bang releases seldom go well, so taking small steps should be the safer approach.
Composing a new service into a Lagom system
Now for the good news. Lagom’s architecture makes it not only possible but practically encouraged for Lagom-built and “other” services to live side by side. This is thanks to its reliance on open standards, open-source software and industry best practices. At a high level, Lagom services are meant to be deployed to platforms like Kubernetes, OpenShift, or cloud-managed infrastructure. Any new service should therefore be deployed the same way.
Next, for communication between services Lagom relies on either HTTP calls (synchronous, request-response) or WebSockets (streaming), serialized with JSON or Protobuf. These are highly standard and “safe” choices. For asynchronous communication Lagom uses Kafka, which in my view can also be called the de-facto industry standard for messaging at scale. So whatever migration target you pick, you will certainly find libraries for communicating over these protocols.
Kafka in particular can prove valuable here. If Kafka is already deployed in your environment, it may be worth using after the migration, even if the migrated service did not previously use Kafka for asynchronous communication. It can be an excellent channel between the migrated service and the remainder of the system, still running on Lagom. Keep in mind that this may also mean changing existing codebases. Consequently, such gradual migrations may call for some intermediate code changes – raising the costs, but lowering the risks too.
Lastly, Lagom promotes the use of third-party service locators (like Consul or etcd. Yet even if you rely on the one shipped with Lagom, you can register your own services in it. Keep in mind that, if the service locator also has to be migrated, this should be done separately from migrating the services.
Migrating service descriptors
Lagom has its own service description language, written as Java or Scala code. Naturally, it will no longer be available after the migration. You may therefore want to pick another way of describing endpoints.
One option is hand-writing OpenAPI or AsyncAPI schemas and generating service stubs from them. You could also go with gRPC specifications with HTTP annotations – with the possible advantage of exposing the service over both HTTP and protobuf protocols. Lastly, a service definition language such as smithy is an option.
Into the cloud
So far we have talked about migrating to a self-hosted service, but bear in mind that a hosted option exists as well. Kalix is the as-a-service successor of the Lagom framework. Its programming model may seem familiar if you have worked with Lagom, although the APIs differ. It runs in the cloud and is billed by usage.
The data model in Kalix revolves around entities, which can be CRUD-like, event-sourced or replicated (with CRDTs). That makes it an even broader choice than what Lagom provides out of the box. Although there are certain limits on how data can be queried, Kalix was built with high-throughput, low-latency use-cases in mind. Moreover, on top of the service abstraction familiar from Lagom, it provides long-running workflows.
A possible difficulty is integrating Kalix-based services into a larger system that includes self-hosted services, particularly when each side uses a different cloud provider (which can matter e.g. due to latency and data transfer costs). Still, here too every interface and communication channel Kalix exposes is based on standards (protobuf or HTTP). Achieving such integration should therefore be fairly straightforward.
Migrating a service
This is likely the heart of the entire migration: how should the services be migrated? There are, however, as many answers as there are services. The simplest case involves services that did not use any of Lagom’s clustering capabilities and were built with the framework merely to keep the tech stack uniform. So for a service exposing a CRUD-like REST or HTTP API, we can actually pick any of the leading libraries or frameworks, according to our target tech stack.
Naturally, it gets harder with services built on the event sourcing/CQRS pattern. Here you need to ask: was event sourcing, with all its tradeoffs, worth it? If yes, great – your best bet is to drop one level of abstraction and use Akka or Pekko Persistence directly. That is what Lagom relies on under the hood. Again, the programming model should be familiar, just with slightly different APIs.
If you choose to abandon event sourcing, the good news is that since the full history is available, filling the new data storage should be fairly easy (which might actually be a good reason to stick with event sourcing?). Keep in mind that other data storage options are available too. Lagom defaults to Cassandra, but Akka/Pekko Persistence provide alternative drivers, e.g. for PostgreSQL. Alternatively, you can move straight to a traditional relational database, either self-hosted or one of the scalable, managed cloud offerings. A compromise between NoSQL-event sourcing and history-less SQL could be SQL-based event sourcing.
To cluster, or not to cluster
A lot of Lagom’s functionality depends on services being deployed within a cluster. This covers scalability and resiliency, where several instances of a service run on many nodes and service discovery routes requests to the right place. Above all, though, clustering is needed for persistence (with event sourcing) and for pub-sub.
If you don’t rely on event sourcing and used Lagom’s clustering solely for scalability and resilience, it may be worth asking whether you really need that extra clustering layer. Service orchestrators like Kuberenetes, if you deploy to them, come with their own clustering layer offering comparable features, such as cluster singletons or load balancing of incoming requests. And every clustering component you can remove from your deployment is a gain – since it makes operations simpler.
As a matter of fact, ShardCake, one of the few alternatives to Akka/Pekko persistence, relies on clustering services provided by Kubernetes rather than its own cluster. This also makes it worth considering when migrating event-sourced services.
Summing up
Unless you fork the framework itself, migrating a fleet of Lagom services will almost certainly mean rewriting code. Still, you can treat the migration as a chance to consolidate or modernize your tech stack, or to divide the system’s functionalities more sensibly into separate services.
We offer technical expertise on every level described above: from the Java and Scala languages and their ecosystems, via architecture and distributed systems consulting, all the way to Kubernetes and cloud deployments. We are recognized for the attention to detail we bring to every project, which makes us a perfect partner to lead organizations through migrations like the ones described here. Get in touch and we will give you our straightforward assessment of what moving off Lagom could involve for your organization!