HOW-TO Build Mesos on Mac OSX – Eclipse

Update – El Capitan upgrade

If you have recently upgraded to El Capitan (OSX 10.11.1) your Mesos build may be broken, due to “missing libapr-1 headers”:

../configure

will cause the following error:

libapr-1 is required for mesos to build.

To resolve, install `apr` using `brew` and then add the `–with-apr` flag to `../configure`:

brew install apr
../configure --with-apr /usr/local/Cellar/apr/1.5.2/libexec

the actual location in your system may vary, depending on how Brew is configured, but the location will be noted during the installation (remember to append libexec).

Preliminaries

  1. Familiarize yourself with the Apache Mesos Getting started guide – or see my guide and gist;
  2. Follow the instructions that I put together to build Mesos on your Mac;
  3. Install Eclipse

At this point we assume that you have a fully built, functioning Apache Mesos distribution on your laptop, on

        export MESOS_HOME=/path/to/mesos

Prepare you Mac OSX Dev Environment

Start by installing XCode and the Command Line Tools via Apple’s AppStore (it will take a while, relax and go grab a coffee or something).

You will need to install homebrew (if you haven’t already done so – run which brew and see what comes up) and then the autotools:

      ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

      brew install git bash-completion maven
      brew install autoconf automake libtool subversion

You will then need to (optionally) install Python 3 (it does not come by default in Yosemite).

While you’re at it, you may want to install Google Test too (this is optional: Mesos distribution brings along its own copy of gtest and will use that on to test itself – make check).

At  this point you should be ready to build and test Mesos:

        ./bootstrap
        mkdir build
        cd build
        ../configure --disable-python --disable-java
        make
        make check

Note – I’ve disabled building the Python and Java extensions as this speeds up compilation significantly (not to mention, there other gory details around getting Maven setup and other Java-specific stuff – another day, another post).

Setup your Project in Eclipse

All going well, we should now be ready to setup Mesos in Eclipse.

(the following is based on the instructions that Bernd shared on Stack Overflow)

  1. Open eclipse. Select “File → Import… → Git → Projects from Git”.
  2. Select “Existing local repository”.
  3. Click on “Add…” and add your local Mesos git directory ($MESOS_HOME). It will show up in the list in the selector box from step 4.
  4. Select the entry for your directory and click on “Next”.
  5. Select “Import as general project” and click on “Next”.
  6. Give your project a name and click on “Finish”.
    Now you have a project.
  7. Make sure you are in the C++ perspective.
    Select/highlight your project in the Project Explorer. Then select “
    File → New… → Convert to a C/C++ Autotools project”.
  8. Edit your project’s properties (Select project, Cmd-I). On the pane “C/C++ Build”, add /build to the value of “Build directory”. It should look like this: ${workspace_loc:/myProjectName}/build
  9. In pane “Autotools” go to “Configure Settings → Advanced”. Under “Additional command line options” you must add -with-svn=/usr/local and can optionally put “–disable-python –disable-java” too, if you don’t want to build Python and Java-related stuff every time.[1]Project Settings


  10. Add /usr/local/include and /usr/local/include/subversion-1 to the “Include directories” in Project Settings (you will need to right-click on the Project’s root node: there is no way to get to this page via Eclipse Preferences)See the image at the bottom of this post for a snapshot of what your includes should look like once all the above is done.

    If you want to build the Java components, add the following to C/C++ Build/Environment:

add MAVEN_HOME = /usr/local/Cellar/maven/3.3.1

(this assumes you have installed Maven via brew  – adjust as appropriate for your environment).

Finally, configure the C++ indexer: Eclipse →  Preferences →  C/C++ →  Indexer: make sure “Enable indexer” is checked; “Index source files not included in the build” and “Index unused headers” must be checked; and, optionally, both “Automatically update the index” and “Update index immediately after file-save” should be checked.

Give its cache plenty of RAM (eg, 640MB)

Now for the fun part: you must configure a bunch of symbolic links to point from /usr/local/include to the right places:

Make sure you have the following entries in your include path (actual path will vary on your system):

export MESOS_HOME=/path/to/mesos
export GTEST_HOME=/path/to/gtest-1.7.0[2]

and create the following symbolic links from /usr/local/include to these targes:

boost -> $MESOS_HOME/build/3rdparty/libprocess/3rdparty/boost-1.53.0/boost
c++11 -> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1
glog -> $MESOS_HOME/build/3rdparty/libprocess/3rdparty/glog-0.3.3/src/glog
google -> $MESOS_HOME/build/3rdparty/libprocess/3rdparty/protobuf-2.5.0/src/google
gtest -> $GTEST_HOME/include/gtest
jni.h -> /System/Library/Frameworks/JavaVM.framework/Versions/A/Headers/jni.h
libltdl -> ../Cellar/libtool/2.4.6/include/libltdl
ltdl.h -> ../Cellar/libtool/2.4.6/include/ltdl.h
mesos
picojson.h[3]
process
stout
subversion-1 -> ../Cellar/subversion/1.8.13/include/subversion-1

TODO: Python or Bash script to build all the above

After you do that, click Ok, you should expand the “Includes” node in the Project tree and see something similar to the one shown above.

Build Mesos

All going well, you should now be able to Build the project (hit the “hammer” icon) and see autotools progress in the Console view (make sure to select CDT Global Build Console from the dropdown in the Console View’s terminal icon.

Build a C++ Framework

Create a new “Hello World” C++ Project, make sure to select the “OSX GCC Toolchain” and then repeat the same steps above to add /usr/local/include to the “Includes” directories; you will then have to add libmesos.la to the libraries for your linker to pick up:

C++ Libraries settings

and, if you have a `Unittests` configuration, you can add gtest (libgtest.a) too.

At this point, the “framework” project can successfully include from <mesos/executor.hpp> and do something like:

#include <mesos/executor.hpp>

class MongoExecutor : public mesos::Executor
{ 
public:
        MongoExecutor() {}
        virtual ~MongoExecutor();
};

without causing any syntax errors.

If you want to try a simple example, you can git clone my sample mongo_fw project:

git clone git@github.com:massenz/mongo_fw.git

this should build against the just-build libmesos.la, and then you can execute (once you have Master/Slave up and running) with:

./mongo_fw `hostname`:5050

Make sure you have a working MongoDB Server installed and that you update the location of the configuration file.

Beware of Eclipse configuring the toolchain to be the “Cross gcc” (instead of OSX gcc) – it won’t build the binaries: see here for more details.


[1] If you see an error related to missing subversion1 headers during the configure step, this is the cause

[2] You can either install Google Test (following the guide here) or point it to the Google Test inside the Mesos tree (under libprocess/3rdparty)

[3] picojson.h must be manually copied to /usr/local/include

Mesos Includes

Advertisements

3 thoughts on “HOW-TO Build Mesos on Mac OSX – Eclipse

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s