Apache Kafka has been on my list of things to understand properly for years, filed next to Spark under "read about, never run". What moved it up the list was KIP-833, which marked KRaft as production ready in the 3.3 release — the point at which a plan to remove ZooKeeper from Kafka stopped being a roadmap item and started being something you could choose.

A small piece of trivia first, because it caught me out while downloading: 3.3.0 was never announced. A significant bug turned up after the artifacts had already been pushed to Apache and Maven Central, so the release was not announced and 3.3.1 went out instead with the fix. If you go looking, 3.3.1 is the release, and 3.3.0 is the one you are told not to use.

What ZooKeeper was actually doing

Kafka's data — the topics themselves — was never in ZooKeeper. What lived there was the metadata about the cluster: which topics exist, how many partitions they have, which broker leads each one, which replicas are in sync, the configuration overrides, the ACLs. ZooKeeper also ran the controller election that decides which broker is in charge of reacting to all of that.

Which means operating Kafka meant operating two distributed systems. A second quorum with its own odd-numbered node count, its own failure modes, its own version compatibility matrix, its own monitoring and its own place in the runbook — in service of a system whose entire pitch is that it is a replicated log you can rely on.

There was a scaling consequence too, and it is the one that shows up in incidents rather than in diagrams. When the controller failed over, the new controller had to load the cluster's metadata out of ZooKeeper before it could do anything. That load is proportional to how much metadata there is, so recovery time grew with the number of partitions. The more you used Kafka, the slower it got at recovering.

What KRaft does instead

The metadata moves into Kafka. Specifically into an internal topic, __cluster_metadata, replicated across a quorum of controller nodes by a Raft implementation built for the purpose.

The reframing is the interesting part, and it is one Kafka already sells to everybody else: metadata stops being a tree of state you fetch and becomes a log of events with offsets. Brokers follow that log the way a consumer follows a topic. A broker that has been away does not reload the world; it asks for what happened after the offset it last saw. A new controller does not rebuild from a cold read, because the quorum already has the log.

Setting one up is where the change becomes concrete. Nodes now have roles:

process.roles=broker,controller
node.id=1
controller.quorum.voters=1@kafka-1:9093,2@kafka-2:9093,3@kafka-3:9093
listeners=PLAINTEXT://:9092,CONTROLLER://:9093
controller.listener.names=CONTROLLER

Running controller and broker in the same process is the convenient shape for a laptop and is not what you would do for anything real, where the controllers are their own small nodes.

The step that surprises everyone coming from a ZooKeeper cluster is that storage now has to be initialised with a cluster ID before a broker will start:

KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"

bin/kafka-storage.sh format \
  --config config/kraft/server.properties \
  --cluster-id "$KAFKA_CLUSTER_ID"

bin/kafka-server-start.sh config/kraft/server.properties

There is no external registry to derive identity from any more, so the cluster has to be given one. Miss that and the broker refuses to start with a message that is, to be fair, perfectly clear.

What "production ready" was not claiming

This is the part I would want someone to tell me before I got excited, and it is stated plainly in the KIP rather than hidden.

KRaft is production ready for new clusters. There is no migration path from an existing ZooKeeper-based cluster in this release. That work is planned for a later 3.x release, which will act as a bridge between the two worlds. If you already run Kafka, this announcement is not something you can act on; it is something you wait for.

There are also features that KRaft mode did not yet support at 3.3, and any one of them is disqualifying if you depend on it:

• JBOD configurations with multiple storage directories per broker.
• Delegation tokens.
• Changing certain dynamic configurations on a standalone controller.

The direction beyond that is not ambiguous — the plan on paper ends with ZooKeeper support removed entirely in Kafka 4.0 — but "the plan says it will go" and "you can move today" are separated by a good deal of release engineering.

So the honest state of things in December: if you are standing up a new cluster and you do not need JBOD or delegation tokens, KRaft is a reasonable choice and removes an entire system from your operational surface. If you have an existing cluster, nothing about your Monday changes.

What I actually took from it

I do not run Kafka in production and this was a weekend with three containers on a laptop, so I am not going to pretend to opinions about it at scale.

What I will keep is the shape of the fix. A system that outsources its consensus to another system inherits that system's operations, its failure modes and its upgrade schedule — and the cost of that shows up years later, in the runbook and in recovery times, not in the original decision. Kafka's answer was to notice that it already had the primitive it needed. The thing it sells to its users is an ordered, replicated, offset-addressable log. Its own metadata problem was a state-synchronisation problem right up until somebody modelled it as a log instead, and then most of the hard parts turned into something the project already knew how to do well.

That is a better lesson than anything about Kafka specifically, and it is why this release was worth a weekend even without a cluster to run it on.