Apache Kafka vs Apache Iggy: One Log, Two Engines
A comparison of Apache Kafka and Apache Iggy that looks at their storage models, durability guarantees, benchmark claims, and how mature their replication is.
Is Apache Iggy simply a faster Apache Kafka?
Each of them stores messages in partitioned, append-only logs. Each has offsets, retention policies, consumer groups, and replay. Apache Iggy is implemented in Rust with a thread-per-core architecture built on io_uring, whereas Kafka runs on the JVM. A benchmark can compare how they perform, as long as we first make sure the tested configurations offer the same guarantees.
In case you haven’t heard of it: Iggy is a message streaming platform written in Rust that graduated from the Apache Incubator to become a top-level Apache project in August 2026.
What precisely does a successful write mean? Was the message copied into an in-memory buffer? Written to the operating system page cache? Flushed to a local disk? Replicated to a different machine? Replicated to a different data center? And what happens if the machine, the rack, or the whole data center vanishes right after the producer gets an acknowledgement?
Release update: Apache Iggy 0.9.0 came out on September 18, 2026. It ships the VSR-based clustered server that had previously only been available in development builds.
A young project, moving fast
First, a little background on Iggy. Its creator launched it in early 2023 to learn Rust by building something real; the name is an abbreviation of Italian Greyhound – a small yet remarkably fast dog. The project joined the Apache Incubator in early 2025 and graduated by unanimous vote in August 2026, at almost the same moment its long-running replication effort was nearing a first clustered release. That effort is now part of the 0.9.0 release.
They agree on the basic data model
Iggy isn’t attempting to swap the log for some other abstraction. Anyone who has used Kafka will find its storage model familiar. An Iggy Stream is an extra namespace sitting above topics. Other than that, the hierarchy closely mirrors Kafka’s. A partition is an ordered, append-only log made up of segments. Messages carry offsets. Consumers can begin at an explicit offset, a timestamp, the start or the end of the log, or resume from a consumer offset stored on the server.

That makes this comparison different from one with systems that swap the event log for a database or a streaming table, such as Apache Fluss. Iggy retains the partitioned log and changes the execution engine, memory management and I/O path instead.
Where Iggy actually differs
Kafka has had plenty of time to tune its storage path. It writes sequentially, batches heavily, relies on the operating system page cache, and can use zero-copy transfer when delivering data to consumers. Batching and sequential I/O cut down the work required per message.
Iggy takes a more hands-on approach to controlling the runtime. From version 0.6.0 onward, the server switched from a Tokio-based model to a thread-per-core architecture based on the compio runtime, which relies on io_uring on Linux. Partitions are mapped to CPU-pinned shards: rather than a work-stealing executor shuffling tasks between threads, every shard owns a slice of the workload and runs it on a particular CPU core.
In addition, Iggy manages its own preallocated memory pool and uses vectored I/O to write batches of message buffers to disk. The goal is largely predictability: in a thread-per-core model, a hot partition doesn’t keep hopping between worker threads, and there is less synchronization on the data path, less cache invalidation, and fewer allocations. The project reports big tail-latency gains from this rewrite, which is one area where work-stealing runtimes can cause trouble.

The JVM-versus-Rust difference accounts for only part of the design. Kafka isn’t a typical application that keeps allocating and transforming message objects on its hot path; much of its performance stems from touching the payload as little as possible and leaving caching and sequential I/O to the operating system. The more meaningful difference is that Kafka hands much of its storage caching over to the OS, whereas Iggy attempts to control more of the execution and memory path on its own.
What happens when a producer writes a message?
On a single node, Iggy’s write path looks approximately like this:

Messages are held in a buffer until a configurable count or size limit is hit – at the time of writing, the defaults are 1024 messages or 1 MiB – and fsync can be configured per topic. An acknowledgement doesn’t necessarily mean the data is in durable storage. Iggy’s documentation states plainly that the default single-node setup offers weak durability guarantees and that a crash or power failure can lose messages unless the durability settings are tuned; the server provides enforce_fsync and flush thresholds to manage that trade-off.
Kafka faces the same fundamental issue but handles it in another way. Kafka doesn’t usually fsync each individual message either. Instead, durability in production generally relies on replication:
| Setting | Value |
|---|---|
| Replication factor of the topic | 3 |
min.insync.replicas of the topic |
2 |
acks of the producer |
all |
Kafka keeps track of an ISR, the set of in-sync replicas. With acks=all, a successful acknowledgement waits for the in-sync replicas; min.insync.replicas defines the smallest ISR size needed for the write to succeed. It doesn’t mean that in this configuration Kafka always waits for exactly two replicas.
This difference is crucial when comparing latency. An Iggy producer waiting for a local write and a Kafka producer waiting for replication to a second machine aren’t measuring the same thing. Nor are a local buffered write, a local fsync, and a write replicated across two data centers. Before we compare numbers, we have to define which failure an acknowledged message is expected to survive.
What do the published numbers actually measure?
I’m intentionally not running my own Kafka-versus-Iggy benchmark for this article, but the figures published by the Iggy project are still worth a look-keeping the previous section in mind.
The project’s FAQ says that the move to thread-per-core alone delivered up to 92% better P9999 tail latency, along with an 18% throughput gain with fsync turned on. The project site promises millions of messages per second with sub-millisecond tail latencies on a single node, and the repository includes its own iggy-bench tool, noting that the default configuration is tuned for performance. These are results reported by the project for the single-node engine under the acknowledgement settings that were tested.
Comparing single-node Iggy with Kafka is possible. A broker-only benchmark published by a commercial Iggy vendor reports higher throughput and lower tail latency. Before relying on those results, check the durability settings along with message size, batching and hardware. A replicated comparison should additionally name the exact release or development build that was tested.
The reported P9999 improvement compares Iggy before and after the thread-per-core rewrite. It isn’t a measurement against Kafka. Crediting the architecture with the gain requires checking the workload and configuration used in both runs.
Single-node Iggy is pleasantly simple
The server is a native binary bundling its own CLI and Web UI; it exposes Prometheus metrics and OpenTelemetry integration, supports several transports, and needs neither a JVM nor a separate metadata service. If what I need is a persistent event log on one powerful server with local NVMe storage, Iggy offers a lot for fairly little infrastructure.
Kafka can of course run as a small cluster, but a production Kafka deployment rests on a different assumption: machines fail, so partitions have replicas and cluster metadata has a quorum. All that extra machinery comes at a cost – and it’s also what lets Kafka cope with failures that a single Iggy process can’t. When comparing deployments, factor in the effort needed to keep the service available after a node goes down.
Replication in 0.9.0
Multi-node clustering built on Viewstamped Replication (VSR) already exists in the new server code. In August 2026, server-ng was promoted to be the main iggy-server, and 0.9.0-edge builds began to appear. The project describes the server side as mostly complete: view changes, state transfer, leader forwarding, client-session recovery, node authentication, TLS between nodes, and deterministic simulation testing of crashes, delays, restarts, and network partitions are all implemented – the FAQ characterizes clustering as maturing toward production readiness. For a deployment, verify that the SDK version you pick supports the clustered server and its VSR wire protocol.
The 0.9.0 release replaces 0.8.0 and ships the clustered server. With it available, replicated workloads can be tested against an official release.
Kafka’s replication model has been running in production for more than a decade. Every partition has a leader and follower replicas:image
Followers copy the leader’s log, Kafka uses the ISR to track which replicas are caught up closely enough, and producer acknowledgement semantics can be bound to that set. Operators have well-documented procedures for replica failures, ISR changes and recovery.
Iggy’s deterministic simulation tests run through failure sequences before deployment. I’d still want operational evidence from replicated workloads, though: recovery times, rolling upgrades and latency while replicas are catching up.
Surviving the loss of a data center
Certain systems have to survive losing a whole data center. Replica placement and controller quorums must take that failure into account.
Kafka’s broker.rack setting isn’t restricted to a physical server rack: its value can stand for an availability zone or some other failure domain, and Kafka’s replica placement distributes a partition’s replicas across those domains. With three data centers close together, one replica of each partition in every one of them, acks=all, and a suitable min.insync.replicas, the producer acknowledgement can rely on replicas outside the local data center. How this differs from asynchronous disaster recovery is clearest when shown side by side:

With asynchronous cross-cluster replication such as MirrorMaker 2, the source can acknowledge messages before they arrive at the second cluster. A stretched cluster can insist on an acknowledged copy outside the local DC, at the price of inter-DC write latency. Replica placement, the controller quorum and recovery settings all have to support the failure scenario you’re planning for.
If the requirement is zero loss of acknowledged data after a site failure, I’d go with a Kafka deployment configured and tested to deliver exactly that. Iggy’s clustering work doesn’t yet offer a comparable multi-DC setup documented for production.
The more detailed topologies include Confluent Platform Multi-Region Clusters.
Consumer groups look much more familiar
Iggy offers server-side consumer offsets and consumer groups: several consumers in a group split the topic’s partitions among themselves, and partitions get reassigned as members come and go. Iggy uses cooperative rebalancing with a pending-revocation phase, so ownership needn’t shift abruptly while a consumer is still wrapping up its work.
The mental model is strongly reminiscent of Kafka:

Consumer groups coordinate who owns which partition as membership changes, so consumers can share the work rather than each reading the whole topic.
Exactly-once is a much bigger difference
For exactly-once processing, we have to state which operations must commit together.
Iggy supports either automatic or manual management of consumer offsets, plus optional message deduplication based on message IDs. Its documentation frames exactly-once as application-level deduplication, with a server-side deduplication cache available to help filter out duplicate IDs. That can be handy, but it’s a different mechanism from Kafka transactions.
Kafka supports idempotent producers and transactions that can atomically combine producing records with committing consumed offsets. This is important for a consume-process-produce application:

A Kafka transaction can commit output records together with consumed offsets atomically. Downstream consumers need read_committed to filter out aborted output. External HTTP calls or database updates fall outside that transaction.
Iggy pushes more of this coordination onto the application. If an application depends on Kafka transactions, message-ID deduplication by itself can’t stand in for them.
Compaction is another difference
Kafka retention isn’t confined to removing old data by age or overall size. A compacted topic can keep the most recent value for each key:
user-1 -> Alice
user-2 -> Bob
user-1 -> Alice
and at some point drop the older user-1 -> Alice record, while still keeping enough history to rebuild the current state. This capability matters for changelog topics, restoring Kafka Streams state, CDC-style datasets, and configuration streams.
Iggy’s present storage documentation covers time- and size-based retention of sealed segments; at the moment it offers nothing equivalent to Kafka’s key-based log compaction. For pure event history that’s irrelevant. For using the log as the durable backing store of materialized state, it’s significant.
Iggy messages do get their own identity
I appreciate that Iggy’s record model includes a message ID. In Kafka, a record’s natural identity is typically the triple (topic, partition, offset). Iggy messages additionally carry a 128-bit message ID, which the optional deduplication mechanism uses and which exists independently of where the message physically sits in the partition. It doesn’t replace transactional semantics, but it’s a handy primitive to have baked into the message model.
The protocol choice is deliberate
Iggy doesn’t currently support the Kafka protocol. It has its own binary protocol, available over TCP, QUIC, and WebSocket, as well as a separate HTTP API, with TCP being the route meant for the lowest latency. This lets the Iggy team evolve the protocol alongside the storage engine – the 0.8.0 release, for instance, brought a major rewrite and consolidation of its wire format. Kafka can’t do that lightly, since its protocol forms part of an enormous compatibility surface relied on by clients and products written in many languages.
Iggy currently offers SDKs in several languages, but the Rust client serves as the reference implementation, and feature coverage varies from language to language. A Kafka wire-protocol proxy is on Iggy’s roadmap and, according to the project, is being developed – should it mature, it could connect existing Kafka producers and consumers to Iggy and change the migration picture significantly. Until that happens, moving from Kafka to Iggy is an application migration, not a matter of swapping brokers.
The same applies to connectors
Iggy already has its own connectors runtime with sources and sinks for systems such as databases, search engines, object storage, and Iceberg. A migration still calls for comparing connectors one by one with Kafka Connect: vendors and open-source projects have had years to build around Kafka’s API and protocol, and that range of integrations is part of the product even though none of the code lives in the Kafka broker itself. Review the operations, delivery guarantees and recovery behaviour of every connector the application relies on.
So is Iggy a Kafka replacement?
For certain workloads, possibly. For every Kafka workload today, no.
If what you need is a persistent local event log with very low latency, straightforward deployment, and high throughput on a single machine, Iggy is a credible and technically fascinating option. Version 0.9.0 also offers an official release for evaluating replicated deployments. Its implementation deserves study whether or not we end up deploying it: thread-per-core execution, explicit shard ownership, io_uring, NUMA awareness, and custom memory management demonstrate what a streaming log can look like when built from the ground up on modern Linux, free of compatibility constraints.
For an existing Kafka application, I’d check whether it relies on:
- transactions and atomic offset commits;
- key-based log compaction;
- client libraries and connectors;
- rolling upgrades and recovery procedures;
- multi-DC availability and data-loss guarantees.
These requirements can rule out a migration before throughput even comes into play.
What I would test next
Next, I’d put Iggy’s replicated path under load: shut down a node mid-write, cut communication between replicas, and measure recovery time and tail latency. After that, I’d confirm which acknowledged messages survive with the chosen configuration.
For a Kafka migration, I’d run the same application workflow on both systems, retries and consumer restarts included. That would reveal whether Iggy meets the workload’s guarantees at an acceptable latency and operational cost.