Code Trips & Tips

Apache Mesos Leader Master Discovery using Zookeeper (part 2)

This is a follow-up on a previous post that describes how to retrieve the URL of the Apache Mesos leading Master, using ZooKeeper as the coordination framework. We will soon release Apache Mesos 0.24, probably shortly before MesosCon: amongst many new features (including a brand new HTTP API) the MasterInfo data that was published to…

Marco
2 minutes

This is a follow-up on a previous post that describes how to retrieve the URL of the Apache Mesos leading Master, using ZooKeeper as the coordination framework.

We will soon release Apache Mesos 0.24, probably shortly before MesosCon: amongst many new features (including a brand new HTTP API) the MasterInfo data that was published to ZooKeeper during the Leader Election round is now being pushed in JSON format, making it much easier to read and parse by non-Protocol Buffer-aware code.

The code in the ZookeeperDiscoveryJson class[1] shows how one would go about retrieving the data from ZK and exposing to a client application (and/or tooling).

[1] A “better” version of this code will soon be posted to our Mesos Commons github repository (it is currently PR#2_) which will also include, soon, the ability to set a “hook” method that will be invoked when there is a change in Leader.

Alternatively, just import this example code (or, better still, wait for mesos/commons to be updated and use that one) and use it like is shown in the bin/mesos-leader example script:

from mesos_commons.discovery import zookeeper_discovery, ZookeeperNoMaster

# Options parsing etc.

def main(cfg):
    logging.info("Connecting to Zookeeper at {}".format(cfg.zk))
    zk_discovery = zookeeper_discovery(cfg.mesos_version, cfg.zk, timeout_sec=15)
    logging.info("Connected")
    try:
        if cfg.all:
            res = zk_discovery.retrieve_all()
            logging.info("Found {} Masters; current Leader: {}".format(len(res), res[0].get('hostname')))
        else:
            res = zk_discovery.retrieve_leader()
            logging.info("Found leader: {}".format(res.get('hostname')))
        print(json.dumps(res, indent=4, separators=[',', ':']))
    except ZookeeperNoMaster as ex:
        logging.error("Could not find any Mesos Master: {}".format(ex.message))

This should be invoked pointing to the same ZooKeeper URL and znode as the Mesos Master --zk option flag, e.g.:

--zk=zk://localhost:2181/mesos/test

and will emit the JSON for the elected leading Master:

{
    "ip":855746752,
    "hostname":"gondor",
    "pid":"master@192.168.1.51:5051",
    "id":"20150806-001800-855746752-5051-509",
    "version":"0.24.0",
    "address":{
        "ip":"192.168.1.51",
        "hostname":"gondor",
        "port":5051
    },
    "port":5051
}

It is worth noting that both Mesos 0.23 and 0.24 understand both protocols (binary Protobuf and JSON) so they are compatible with each other (and 0.23 is also, as usual, compatible with 0.22).

Leave a comment

Trending