Interview Bootcamp
Vote

0% completed

​

Role of ZooKeeper

A broker fails in the middle of the night. Something has to notice it is gone. Something has to pick new leaders for the partitions it held. Then every other broker has to be told what changed.

That work is called cluster coordination. This lesson covers how Kafka does it.

What the coordination layer is for

Kafka needs one agreed picture of the cluster. That picture is the cluster metadata. It holds a small set of facts:

  • which brokers are alive right now
  • which topics exist, and how many partitions each one has

.....

.....

.....

Like the course? Get enrolled and start learning!
J

Junaid Effendi

· 4 years ago

In newer version, Kafka broker gets the metadata from Zookeeper, does that mean Kafka broker is now maintaining a state and cannot be considered stateless anymore?

A

Alexey Pirogov

· 2 years ago

ZK isn't required anymore and offsets not stored in ZK for many versions.

Show 1 reply