Estimate project
← All posts

Apache Kafka 4.0.0 Is Out: KRaft, Queues, and Faster Rebalances

Explore the new features and performance gains in Kafka 4.0 and find out how these changes can improve your data streaming setup.

3 min read
KafkaApache KafkaTechnology News

Kafka 4.0 at a glance

In March 2025, Apache Kafka hit an important milestone with the release of version 4.0.0. The release brings new features, enhancements, deprecations, and significant updates. Let’s go through the key points of Apache Kafka 4.0.0 and look at what these changes mean for developers, system architects, and integrators.

Farewell ZooKeeper, Long Live KRaft

For years, running Apache Kafka meant also running an external ZooKeeper cluster to handle cluster metadata and tasks such as leader election. This extra component made operations more complex, both in production and in local development. That era has now officially come to an end.

KIP-500 – first shipped as an early access feature in Kafka 2.8.0, released in 2021 – four years ago already! In the releases that followed, KRaft (Kafka Raft) matured, became production-ready, and gained migration features that made it fit for real-world deployments.

Kafka 4.0.0 drops ZooKeeper support entirely. KRaft is now the default – and the only – mechanism for managing metadata in Kafka, which substantially changes how Kafka clusters are designed and run.

If you plan to move from a ZooKeeper-backed cluster to a KRaft-based one, make sure to read the official migration guide to learn about the limitations, tooling, and best practices.

KIP-848: Smarter and Faster Rebalances Thanks to the New Consumer Group Protocol

KIP-848 brings the next generation of Kafka’s consumer rebalance protocol, fundamentally changing the way consumer groups get partitions assigned.

This matters a great deal for large-scale deployments, where redeploying applications or restarting consumers often triggers rebalances that hurt system performance by stopping message processing until the whole rebalance round completes.

What is different?

  • Partition management and assignment now happen on the broker, which takes over duties that used to belong to the consumer group leader.
  • The internal logic of the Kafka client is much simpler.
  • A new member joining a consumer group can receive new partitions incrementally. Previously, it had to wait until every member revoked/rebalanced its partitions, which needlessly increased consumer latency.

The new protocol aims to boost performance considerably and to cut downtime and latency, particularly in large deployments.

You can also watch the Confluent presentation on this subject from Confluent Current 2024.

KIP-932: Queues in Apache Kafka 4.0 – Early Access Preview

Kafka’s messaging model, built on topics and partitions, has become a foundation of modern event-driven architectures. Some use cases, however, call for more traditional queue semantics, like those found in RabbitMQ or Google Cloud Pub/Sub. Processing messages out of order is handy when handling one message may fail temporarily and has to be retried. Over the years, teams have built many custom and complex retry mechanisms to meet these needs.

KIP-932 brings a preview of the Queues concept to Kafka 4.0, opening the door to kinds of workloads that weren’t possible before.

What it brings:

  • Shared groups, which allow cooperative consumption of messages.
  • Support for individual message acknowledgments and redeliveries.
  • Unordered processing, helpful when some messages can fail temporarily and need to be retried.

This marks a big leap in Kafka’s development, turning it into a realistic option for use cases that used to need a separate queuing system.

Removal of Deprecated APIs and New Java Version Requirements

Major releases are a chance to tidy up, and Kafka 4.0.0 takes it by removing deprecated APIs and modernizing the platform.

Key Updates:

  • Java 8 is no longer supported.
  • Kafka Clients and Streams now need Java 11+.
  • Kafka Brokers, Connect, and Tools need Java 17+.

This brings Kafka in line with modern Java runtimes, which provide better performance, GC algorithms, and long-term support. Java 11 has been around since 2018 and Java 17 since 2021, so both versions are stable and broadly adopted.

One more important change: old protocol API versions are gone. Kafka 4.0 Java clients now need brokers on version 2.1 or later. Support for message formats v0 and v1 has also been removed, which may affect older producers or consumers.

Summary

KRaft, the new rebalance protocol, and Kafka Queues are the three game-changing updates in Kafka 4.0. It’s good to see the platform evolving actively, with a clear emphasis on simplicity, performance, and new use cases.

Commercially, Confluent Platform 8.0 (built on Kafka 4.0) is expected in summer 2025, and it will probably bring many of these features into enterprise-grade products.

Planning a Kafka 4.0 upgrade or looking into Kafka Queues and need support?
As a Confluent Premium Partner, our team has in-depth Kafka expertise and can help you along the way – from migration and tuning to architecture design.