Comments on: Building a Distributed Log from Scratch, Part 2: Data Replication

By: Aravind

Aravind — Fri, 24 Dec 2021 02:53:13 +0000

Regarding these 2 sections of text

> The thought here is if we are replicating to enough nodes, the replication itself is sufficient for HA of data since the likelihood of more than a quorum of nodes failing is low, especially if we are using rack-aware clustering. …

AND

> Another caveat with Kafka is unclean leader elections. That is, if all replicas become unavailable, there are two options: …

Why is it that we’re worried about all replicas becoming unavailable in Kafka for leader elections, but not as much from the replication/durability perspective?

By: Tyler Treat

Tyler Treat — Tue, 23 Jan 2018 22:03:21 +0000

Nice catch, thanks.

By: Chris Young

Chris Young — Tue, 23 Jan 2018 21:39:49 +0000

“By default, Kafka favors availability by choosing the second strategy.” — you meant the *first* strategy here — picking a replica not necessarily in the ISR set? But actually, since 0.11, unclean.leader.election.enable defaults to false, so please update that paragraph :)

By: Ming

Ming — Thu, 04 Jan 2018 03:10:23 +0000

Hi, are you trying to actually build a distributed log in this serials of blog post?

By: Tyler Treat

Tyler Treat — Thu, 28 Dec 2017 15:37:44 +0000

Yes, the MVP currently does not address the Raft scalability problem. These are some potential solutions but not yet implemented. This goes for the dual writes in the Raft log. However, with this approach, reads would still be sequential from the log (as they are now). These will be two big areas for improvement following an initial release.

By: Byron Ruth

Byron Ruth — Thu, 28 Dec 2017 12:33:56 +0000

Great article Tyler. Two clarifications I have.

First, you first stated that NATS Streaming currently uses one Raft Group per topic, but then mentioned the problem with this (explosion of network traffic). Is the plan to adopt the MultiRaft approach the Cockroach Labs folks developed?

Second, in the diagram of the message log being an index of the logical offset which points to the Raft log physical offset, I am assuming this implies the physical message (bytes) are stored only in the Raft log? What is the cost of dereferencing the message while reading the message log? I am making the assumption messages are currently read sequentially from the log files and this approach introduces a new indirection.

By: Shiju Varghese

Shiju Varghese — Thu, 28 Dec 2017 06:50:19 +0000

Brilliant article.