Eligible Leader Replicas (ELR) in Kafka 4.1: The Essentials
A detailed walkthrough of KIP-966 and Eligible Leader Replicas (ELR) in Apache Kafka 4.1. See how distinguishing in-sync replicas from replicas that can safely take leadership closes the Last Replica Standing edge case and stops data loss after unclean broker shutdowns.
Edge cases are everywhere in distributed systems, and Kafka is no different - there used to be an extremely unlikely sequence of events in which even committed messages could disappear. This prompted KIP-966 to introduce Eligible Leader Replicas (ELR) - a mechanism that deals with leader eligibility by distinguishing “in-sync replicas” from “replicas that can safely become leader”. Let’s take a closer look!
How Kafka Could Lose Committed Data
I assume here that you know the basics of Kafka and understand its durability guarantee.
KIP-966 addresses an extreme edge case that could genuinely break Kafka’s durability guarantees. Even with a correct configuration and the strictest producer setting, acks=all, committed messages could still be lost when an unclean shutdown coincides with the last replica standing problem. The chance of running into exactly this sequence is tiny, but the risk was serious enough to justify improving Kafka’s failover logic. To understand what drove KIP-966, let’s go through the following scenarios in detail:
The unclean shutdown scenario
On a graceful shutdown, Kafka flushes its state, writing any data buffered in memory and in the operating system’s page cache to disk, and releases all resources cleanly. An unclean shutdown, by contrast, happens when a broker is killed abruptly, for instance by a power cut or a server machine failure. In that case, some records may be sitting in the Kafka operating system’s page cache without having been persisted to disk yet. Kafka does not flush to disk before treating a message as committed - doing so would be extremely costly - rather, it flushes asynchronously in batches. Settings such as log.flush.interval.messages or log.flush.interval.ms exist, but the default advice is to leave them unset and rely on the operating system’s background flushing, since that is more efficient.
As a result, after an unclean shutdown and subsequent recovery, the broker may return with a shorter log than it had before the crash.

In normal operation - when the broker has other healthy, in-sync replicas (ISR) available - Kafka deals with this smoothly by making one of the remaining healthy, in-sync replicas (ISR) the leader. Once the crashed broker rejoins the cluster, its log segments are automatically synced to match the current leader’s state.
The Last Replica Standing problem
With an unclean shutdown, there is one extreme edge case in which committed data can be lost. It occurs in the Last Replica Standing (LRS) state - when the broker hit by the unclean shutdown holds the only replica remaining in the ISR. This can happen when the other replicas drop out one after another within a very short time, for example because of network problems.
You may be thinking: “With acks=all we’d certainly also set the min.insync.replicas parameter - so with just 1 replica left in-sync, Kafka should refuse to accept data”. That’s correct - but only for new data. The actual problem concerns data that was already committed and acknowledged (while all brokers were healthy) but had not been flushed to disk yet.
The old Kafka behaviour never lets the ISR list become empty - so here, once the whole cluster is back online, the leader that just crashed (the last standing replica) is still considered the sole valid source of truth and is elected leader again. Since it lost its page cache in the unclean shutdown, the healthy replicas - which actually have the complete, intact committed data safely stored on disk - have to replicate from it and truncate their own correct logs.
How can a replica be fully up to date and yet outside the ISR? A replica can drop out of the ISR not only by falling behind, but also by being temporarily unreachable - because of network problems, long JVM garbage collector pauses, short disk I/O stalls, and so on. Any of these can interrupt the fetch requests sent to the leader. If a follower replica does not send a fetch request within the window set by replica.lag.time.max.ms, the leader drops it from the ISR to keep the active replication quorum healthy, even though the replica’s log has fully caught up.
To stress it once more, this is an extreme edge case. For such global data loss to occur, everything has to fail at the same time:
- Network problems knock brokers out one at a time until only one remains (this can take milliseconds).
- That very last broker (in LRS state) goes through an unclean shutdown before its cache is flushed to disk (this takes seconds).
- The healthy brokers come back to the cluster but must discard their correct data to match the corrupted leader’s state.
Given this specific Kafka setup:
- three replicas (brokers 0, 1, 2)
min.insync.replicas=2- producers configured with
acks=all.
the sequence may look like this:

The main changes in KIP-966
To solve the “last replica standing” problem, KIP-966 divides the duties of the former In-Sync Replicas (ISR) list. Before looking at the new replica states, though, we need to cover an essential precondition: a required fix to Kafka’s High Watermark mechanism.
Fixing how the high watermark advances
KIP-966 brings a small fix to how Kafka advances the High Watermark (HWM) - the last offset committed across all in-sync replicas. Consumers, for example, use the HWM to make sure they never read data that has not yet been safely replicated.

Source: Kafka The Definitive Guide
Since KIP-966, the High Watermark may advance only when the ISR size >= min.insync.replicas.
The reason is that Kafka does not restrict a topic to one particular acks configuration - acks is a producer setting, not a broker one. Several clients can write to the same Kafka topic at the same time with different configurations.
Prior to this change, when the topic’s ISR fell below min.insync.replicas:
- Producers using
acks=allwere blocked. - Producers using
acks=0 or 1could continue writing.
In our “last replica standing” scenario, this meant the solitary leader went on accepting acks=1 writes and moving the High Watermark forward on unsafe data. Had Kafka later tried to elect a healthy backup broker to protect the acks=all data, that backup would lack the latest acks=1 messages, so the HWM would have to move backward, breaking consumers.
KIP-966 solves this by turning min.insync.replicas into a global rule for the High Watermark, whatever acks setting the producer uses. This HWM rule is the default starting with Kafka v4.0.
Eligible Leader Replicas
Before Eligible Leader Replicas, a Kafka replica could be in just two states: inside the ISR (trusted) or outside it (untrusted).
ELR sits between the two. It arrived as an experimental feature in Kafka v4.0 and became the default in v4.1 (and therefore applies only to KRaft, not ZooKeeper) - it is a secondary list of replicas that were dropped from the ISR while the quorum was below min.insync.replicas. This usually happens because of network partitions or fencing - a safety mechanism that isolates an unresponsive broker so it cannot make conflicting updates. Although they are not in the ISR, these replicas had data up to the high watermark at the moment they were added to the ELR.

In the words of KIP-966:
“What we ultimately want is to be able to elect a leader without losing data or moving the HWM backward as long as there are at most min.ISR-1 unclean shutdowns.”
To sum up, the responsibilities of the original ISR are now split, and every replica is in one of these states:
- ISR (In-Sync Replicas): As before, with
acks=all, it serves as the active quorum for replicating data. The High Watermark moves forward only up to the point that every ISR member has written. - ELR (Eligible Leader Replicas): Serves as a fallback list for leader election, holding replicas known to be safe leader candidates relative to the frozen High Watermark.
- Out of ISR/ELR: Untrusted replicas that lag behind and cannot be elected leader without risking data loss.
It is worth stressing again: ELR exists only when ISR < min.insync.replicas. In every other case, the ELR list is cleared and remains empty.
Further improvements in KIP-966
Let’s look more closely at the other improvements.
Recognising an unclean shutdown
KIP-966 also makes Kafka better at telling a clean shutdown from an unclean one. Currently, during a graceful shutdown, the broker flushes its logs and writes a CleanShutdownFile to disk - a small JSON file holding the broker’s current epoch. At the next startup, the broker reads this file and includes the saved epoch in its registration request to the KRaft controller. If the epoch matches the controller’s record, the shutdown was clean. If the file is absent or the epoch differs, the controller treats the shutdown as unclean.
Why is this relevant to ELR? When a broker registers following an unclean shutdown, the controller takes it out of both the ISR and the ELR before letting it back into the cluster. This ensures that a broker which may have lost page-cache data cannot quietly stay a trusted leader candidate.
The new partition leader election
With ELR turned on, the controller first searches the ISR for a safe leader and then the ELR. If neither offers an available candidate, the existing unclean.leader.election.enable setting decides what happens next:

KIP-966 also foresees a new unclean.recovery.strategy with Aggressive, Balanced, and None options, but it has not been implemented yet (as of Kafka 4.3 today) and has no target release version, so we will leave it aside for now.
The ISR may now be empty
The old Kafka behaviour did not allow the ISR to become empty. KIP-966 permits an empty ISR. Once the ISR hits zero, the ELR may contain replicas known to hold all committed data up to the High Watermark. Leader election can then pick one of them rather than relying on a broker that was the last ISR member but may have lost unflushed data in an unclean shutdown.
The impact of ELR and KIP-966
KIP-966 is largely an internal durability fix, but it does bring behavioural changes you might notice. Let’s examine them from several Kafka angles:
From the producer’s point of view
Producers with acks=all work as they did before - they get blocked when the ISR falls below min.insync.replicas. The significant change concerns acks=0 and acks=1: because they disregard the min.insync.replicas param, the leader still accepts their writes, but the HWM stops advancing while the ISR is degraded. If producers with different acks settings write to the same topic, this closes a hole through which acks=1 traffic could formerly push the HWM forward on an undersized ISR, contradicting the durability expectations of acks=all.
From the broker’s point of view
Brokers are largely unaware of ELR - the KRaft controller keeps it entirely in partition metadata, and replication still follows the ISR as before.
What changes is the way Kafka deals with broker restarts. On a graceful shutdown, the broker flushes its logs and stores its epoch in the CleanShutdownFile. When it starts up again, the controller uses that epoch to decide whether the shutdown was clean. If it detects an unclean restart, it takes that broker out of both the ISR and the ELR.
From the consumer’s point of view
Consumers are the clients that feel the change most visibly. They can see only messages below the HWM. When the ISR size drops below min.insync.replicas, the HWM freezes - so even freshly produced acks=0/1 messages stay above the HWM and cannot be read until the ISR recovers. Before KIP-966, consumers could go on reading new messages during partial replica outages. With KIP-966, a degraded ISR in effect pauses what consumers can see, not just the durability of what producers write.
The Last Replica Standing problem - with ELR
Let’s run the scenario from the beginning again with the same Kafka setup, this time with the ELR mechanism turned on:
- three replicas (brokers 0, 1, 2)
min.insync.replicas=2- producers configured with
acks=all

Under the old behaviour, Kafka kept broker 2 in the ISR, so after its restart it became leader again. Brokers 0 and 1 - holding the correct copies - had to truncate their logs to match the corrupted leader.
With ELR, things go differently:
When broker 0 drops out of the ISR, min.insync.replicas=2 is still met - so it is not added to the ELR. Even though it had replicated up to the HWM when it left, the rest of the ISR can keep moving the HWM forward, so Kafka can no longer be sure that broker 0 holds everything up to the HWM’s later value.
ISR: [0, 1, 2] → [1, 2]
ELR: [] → []
When broker 1 drops out, the ISR goes below min.insync.replicas - and the HWM stops moving. Kafka knows broker 1 holds every record up to this frozen HWM, so it adds it to the ELR.
ISR: [1, 2] → [2]
ELR: [] → [1]
Once broker 2 crashes and gets fenced, the ISR may end up empty. When the network partition is resolved and brokers 0 and 1 are unfenced, the controller picks broker 1 from the ELR as leader and returns it to the ISR. When broker 2 rejoins later, the controller notices its unclean shutdown and syncs it with the current leader, broker 1.
Even in this extreme, multi-layered edge case, Kafka recovers, and committed acks=all data is preserved.
Wrap-up
What makes KIP-966 interesting is that it neither speeds Kafka up nor adds a new visible feature that users deal with every day. Most users will never run into the Last Replica Standing scenario in production - yet it could still break Kafka’s strongest durability guarantees.
ELR silently closes an extreme edge case and introduces an important distinction into Kafka: a replica does not need to be actively taking part in replication to be the safest choice for leader.
It is one more step toward making Kafka’s durability guarantees hold not just in everyday operation, but also under the rarest combinations of failures.