In-sync replicas and replication factor in Confluent Stretched Cluster 2.5 explained
Find out how to set in-sync replicas (ISR) and replication factor (RF) in Confluent Stretched Cluster 2.5 for the highest data durability, resilience across datacenters, and fault tolerance. See how replication settings, observers, and automatic observer promotion affect availability and performance in comparison with open-source Kafka.
When Kafka is deployed across several datacenters, keeping data durable and available in the face of failures calls for a careful trade-off between consistency and performance.
The Confluent Stretched Cluster 2.5 architecture brings in mechanisms like observers and automatic observer promotion (AOP) that deliver resilience across datacenters while keeping operational complexity low.
In this post, I explain what replication.factor (RF) and min.insync.replicas (min.ISR) do in a Confluent Platform Stretched Cluster, how they influence availability, and in what ways Confluent’s implementation differs from open-source Apache Kafka.
What Is a Stretched Cluster 2.5?
A stretched cluster 2.5 is a Kafka deployment in which brokers run in two fully functional data centers (DCs), while a third, lightweight site runs only metadata services – traditionally ZooKeeper or, in recent Confluent Platform versions, a KRaft controller quorum.
Brokers host partition replicas, and every replica takes on one of three roles:
- Leader – serves client requests (producers/consumers) for a particular partition.
- Follower – keeps a synchronous replica of the partition.
- Observer – keeps an asynchronous replica that, by default, is not part of the ISR (Confluent Blog).
The ISR (In-Sync Replicas) list contains the brokers that are completely caught up with the leader.
The min.insync.replicas setting specifies how many ISR members have to acknowledge a write (with acks=all) before it counts as committed.
To work with observers and stretched clusters, you need to set the number of replicas when you create a topic and then provide confluent.placement.constraints, which decides where every replica goes and which role it takes.
Different settings scenarios
Scenario 1 – RF = 4, min.ISR = 2
Suppose we want 4 replicas of each partition (RF=4), two in each DC. With min.isr = 2, a write counts as successful as soon as two ISR members acknowledge it – even when both acknowledgments come from brokers located in the same DC.
To make sure data is acknowledged in both DCs, you can configure the topic like this:
- DC 1: Leader + Observer
- DC 2: Follower + Observer
Here is the replica placement policy:
{
"version": 2,
"replicas": [
{
"count": 1,
"constraints": {
"rack": "dc1"
}
},
{
"count": 1,
"constraints": {
"rack": "dc2"
}
}
],
"observers": [
{
"count": 1,
"constraints": {
"rack": "dc1"
}
},
{
"count": 1,
"constraints": {
"rack": "dc2"
}
}
],
"observerPromotionPolicy":"under-min-isr"
}
When the DC with the leader goes down, Automatic Observer Promotion (AOP) can move a caught-up observer into the ISR and elect it as the new leader. How long promotion takes depends on replication lag and the state of the network; there is no fixed duration. While this switch is happening, producers using acks=all are unable to send new messages.
This setup is also vulnerable to overlapping failures (for instance, an outage of one DC coinciding with a broker restart in the remaining DC).
Scenario 2 – RF = 4, min.ISR = 3
Here the replication factor is the same as in Scenario 1, but min.isr=3. In this configuration every replica is synchronous – no observers are involved.
Here is the replica placement policy:
{
"version": 2,
"replicas": [
{
"count": 2,
"constraints": {
"rack": "dc1"
}
},
{
"count": 2,
"constraints": {
"rack": "dc2"
}
}
]
}
As a result:
- Every write has to be acknowledged by three brokers.
- Writes go through normally as long as no fewer than three brokers are healthy.
- When a whole DC goes down, just two ISR members are left, and producers using
acks=allwill block until the failed DC comes back ormin.ISRis temporarily reduced to 2.
This configuration favors consistency at the cost of availability when an entire DC fails.
Scenario 3 – RF = 6, min.ISR = 3
With RF=6, every DC holds two synchronous replicas plus one observer:
- DC 1: Leader, Follower, Observer
- DC 2: Two Followers, Observer
Here is the replica placement policy:
{
"version": 2,
"replicas": [
{
"count": 2,
"constraints": {
"rack": "dc1"
}
},
{
"count": 2,
"constraints": {
"rack": "dc2"
}
}
],
"observers": [
{
"count": 1,
"constraints": {
"rack": "dc1"
}
},
{
"count": 1,
"constraints": {
"rack": "dc2"
}
}
],
"observerPromotionPolicy":"under-min-isr"
}
This layout gives strong durability and greater resilience:
- Restarting or upgrading a single broker causes no downtime.
- When a whole DC fails, AOP promotes a caught-up observer in the remaining DC, bringing availability back.
That makes 6 replicas with min.ISR = 3 the most robust and operationally stable setup for stretched clusters.
Confluent Platform vs. Open-Source Kafka
Confluent’s platform builds on Kafka’s durability model with features like asynchronous observers, stretched clusters (2.5-DC architectures), and replica placement constraints. Fully available since Confluent Platform 6.1, these features allow automatic failover and stronger data consistency across regions. Open Source Kafka’s model has only leaders and synchronous followers, with a conventional replication factor.
Conclusion
For a Confluent Stretched Cluster 2.5 deployment, 6 replicas with min.ISR = 3 and a suitable placement policy strike the best balance between availability and durability: writes are safely replicated to both DCs, and the business keeps running during outages.
Simpler setups such as RF = 4, min.ISR = 2 can work when conditions are ideal, but they may result in data loss or downtime when broker and DC failures occur together.
Confluent’s distinctive features – Observers with Stretched Cluster 2.5 – provide capabilities that open-source Kafka lacks, offering safer failover and stronger data consistency for mission-critical environments.