News and notes
Company news, announcements and engineering notes.
ScyllaDB Beyond Cassandra, Part 2: Search, CDC and the Storage Layer
A technical look at how ScyllaDB and Apache Cassandra differ in secondary indexes, vector and full-text search, change data capture, and workload prioritization, illustrated with a support-ticket example.
Kafka Was Born at LinkedIn. What Did LinkedIn Change Once Kafka Stopped Being Enough?
A look at LinkedIn's Northguard and Xinfra systems and at how they rework Kafka-style log storage, replication, and metadata to overcome scaling limits tied to partitions.
ScyllaDB Is More Than Cassandra Written in C++ – Take a Second Look
ScyllaDB began life as a faster reimplementation of Cassandra. Tablets, Raft, strong consistency and DynamoDB compatibility reveal how far it has grown past that model.
Cassandra Has Changed a Lot Since 3.11 – You May Not Recognize It Now
A look at the architectural changes in Apache Cassandra 5.0 and the features coming in 6.0, including indexing, compaction, vector search, repair and transactions.
Diskless Kafka: What Changes When the Data No Longer Belongs to the Brokers?
A look at diskless Kafka architectures that compares how KIP-1150, WarpStream, AutoMQ, and Redpanda approach keeping Kafka's active log in object storage.
Apache Kafka vs Apache Iggy: One Log, Two Engines
A comparison of Apache Kafka and Apache Iggy that looks at their storage models, durability guarantees, benchmark claims, and how mature their replication is.
Comparing TigerBeetle and PostgreSQL Performance: Benchmark Harness and Cloud Tests
An in-depth performance benchmark that pits TigerBeetle against PostgreSQL in a multi-node Google Cloud Platform environment. It examines double-entry bookkeeping workloads at different request rates and levels of account contention (Zipfian distribution), looking at metrics such as mean throughput, latency percentiles, and the share of dropped requests.
Start with a Log or with a Table? Apache Kafka, Fluss, and Streaming Tables
Comparing, at the architecture level, log-first streaming systems (Apache Kafka) with table-first storage abstractions (Apache Fluss), looking at CDC workloads, duplicated state, and integration with the lakehouse.
Thank you, Halyard & Keel: our internal HR system is complete
Halyard & Keel has finished Kamurj HR, our bilingual internal app for people, projects, time off, hiring and reviews. A short look at what it does and why we needed it.
Kafka and AI stroll into a bar...
Find out how Kafka, AI, and the Model Context Protocol (MCP) fit together. See how to configure mcp-confluent with Claude Code, manage clusters, and get past the pain points of the CLI.
Eligible Leader Replicas (ELR) in Kafka 4.1: The Essentials
A detailed walkthrough of KIP-966 and Eligible Leader Replicas (ELR) in Apache Kafka 4.1. See how distinguishing in-sync replicas from replicas that can safely take leadership closes the Last Replica Standing edge case and stops data loss after unclean broker shutdowns.
Are Apache Kafka Replica Followers Synchronous or Asynchronous?
A detailed look at how Apache Kafka replication works compared with its commit semantics, clarifying how ISR followers differ from Confluent observers.
The Adapter Pattern in Rust: Getting Around the Orphan Rule Using Newtypes and Extension Traits
This article looks at the Adapter pattern in Rust as a practical way of dealing with the Orphan Rule – the language restriction that forbids implementing external traits for external types. It begins by explaining clearly why the Orphan Rule exists and which compilation problems it leads to, then shows step by step how to build a newtype-based adapter, how to turn it into one that holds a reference with explicit lifetimes, and how to wrap everything in an extension trait to get a clean, ergonomic API. A real-world example connects the axum and eyre crates, demonstrating how several specialized adapters can map errors onto particular HTTP status codes. The article also discusses design questions such as Deref and AsRef implementations, public vs. private inner fields, and situations in which third-party options like serde's remote syntax may suit better than a hand-written adapter.
Apache Kafka Disaster Recovery and Multi-Region Architectures: A Guide
Today's systems are event-driven, operate in real time and are spread across the globe.
The Workhorses of the JVM: Stubs and Intrinsics
This article describes what stubs are and why they matter so much inside the JVM.
JEP 531 - Lazy Constants, the journey goes on
A look at the Lazy Constants feature from JEP 531 in Java 27: API changes, error handling, lazy collections and known limitations.
Compaction & Retention: Corner Cases That Distort Your Kafka Lag Metrics
Can you trust your Kafka lag dashboard? Find out how log compaction and retention can mask huge consumer delays and how to spot misleading time lag metrics. Teams frequently rely on Kafka lag metrics to track system health, yet log compaction and retention policies can quietly remove messages before anyone processes them. This article explains why the usual time lag formula – $current\_time - timestamp\_of\_message\_at\_committed\_offset$ – underestimates actual latency once the requested offset is gone. We describe how "klag-exporter" spots these inaccuracies by comparing requested and actual offsets as well as low watermarks. Lastly, we offer practical mitigation strategies, such as tuning min.compaction.lag.ms, optimizing consumer resources, and adding warning indicators to dashboards for more resilient Kafka monitoring.
Say Hello to Panama
With JEP-454, the Foreign Function & Memory (FFM) API became a stable feature. Starting with JDK 22, developers can use this API with no extra configuration or "hassle." This article looks at how the FFM API makes working with native libraries simpler and improves performance.
Rust Patterns in Depth vol. 1: Newtypes in Rust
An in-depth technical look at the Newtype pattern in the Rust programming language. The article examines how newtypes differ semantically from type aliases and explains how wrappers with private fields can enforce domain invariants. It covers newtypes as resource handlers for graph structures, applying the typestate pattern to data validation (e.g., Email and Filter types), and ways of hiding implementation details to stay semver compatible. It also warns about typical pitfalls, like 'hidden conversions' during serialization and excessive boilerplate in simple data structures.
Cutting Time-to-Insight in Company Financial Data by 90%
How the people at our company stopped writing complicated SQL queries and started asking questions in plain English.