Motivation

When deploying a Mesos cluster, made of several Master and Worker nodes, it may not be possible (or desirable) to know the IP address(es) of the Master nodes in advance; however, as they can be deployed using Zookeeper as backing storage, we can use the information stored there to retrieve the IP address of the Leader (and the other Masters too).

Background

When launched in HA mode, Mesos master nodes register with Zookeeper and then go through a simple “leader election” contest round: the first one to register (as decided by the lowest-numbered sequence znode) becomes the elected leader.

Hitting any of the masters’ HTTP endpoints will re-direct (302) to the elected leader’s URL – the information about the elected leader, its hostname and IP address are kept in a Protocol Buffer structure that is the data content of the sequential (ephemeral) znode created during the election round.

Approach

Given the above, our approach is simply to connect to the Zookeeper ensemble and, armed with the knowledge of the path to the Mesos masters’ znode (this must be provided when the masters are launched – see the Apache Mesos guide for more details) we can just retrieve the data from the sequential znodes there and deserialize it into Protocol Buffers.

The code in the zk-mesos repository is a Python implementation of the suggested approach.

See the README file there for instructions and more details about how to install it and the actual code.

Future

We are working on implementing a more friendly JSON data format for serializing the MasterInfo data to Zookeeper (see MESOS-2340) and I also have running prototype code that implements this (for release 0.24 – which is one-away from the next one: 0.23) as we need a compatibility transition path.

Until then, the approach provided here is simple and effective.