Putting XGBoost under OS X

XGBoost - C ++ library that implements gradient boosting methods, which can be found more and more often in the descriptions of the winning algorithms on Kaggle. For use from R or Python, there is a corresponding binding, but the library itself must be assembled from source. When I ran make, I saw a lot of errors reporting missing drivers and unsupported OpenMP. Well, not the first time.

Progress does not stand still, and the algorithm is somewhat simpler:

1. / usr / bin / ruby ​​-e "$ (curl -fsSL raw.githubusercontent.com/Homebrew/install/master/install)"
2. brew install gcc --without-multilib
3. pip install xgboost
4. Conquer Leaderboard Tops on Kaggle

From the fields it is reported that when you start XGBoost, it can swear at the missing library /usr/local/lib/gcc/5/libgomp.1.dylib. In this case, you should find it and put it on the specified path.

Previously, it was necessary to do the following:

1. Download Xcode
2. Install command line tools
3. Build Clang with OpenMP support
4. Build the Intel OpenMP library
5. Register the path to the OpenMP library and the corresponding headers
6. Build XGBoost
7. Install Piping for Python
8. Conquer Leaderboard Tops on Kaggle

1. Download Xcode
XCode can be downloaded for free from the App Store. After installation, typing gcc -v on the command line of the terminal, we should see something like the following on the screen:

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer//usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn) Target: x86_64-apple-darwin14.3.0 Thread model: posix 

2. Install the command line tools
If you skip this step, the compiler will not be able to find the standard C and C ++ libraries. In the terminal you need to run

 xcode-select --install 

and follow the instructions.

3. Build Clang with OpenMP Support
This version of the compiler supports OpenMP instructions for parallelization and is developed by Intel employees. Hopefully, one day this branch will flow back into the trunk, and OpenMP will be available in the original Clang from the box. Apparently, some time ago it was possible to install clang-omp with the help of brew, but this happy time has passed. So, we compile the compiler:

 mkdir clang-omp && cd clang-omp git clone https://github.com/clang-omp/llvm git clone https://github.com/clang-omp/compiler-rt llvm/projects/compiler-rt git clone -b clang-omp https://github.com/clang-omp/clang llvm/tools/clang mkdir build && cd build cmake ../llvm -DCMAKE_BUILD_TYPE=Release make -j 4 

If there are more than 4 cores on a machine, it makes sense to correct the figure in the last command.
')
4. Build the Intel OpenMP library
The library for supporting OpenMP is also built from source. Download , unpack, collect:

 mkdir build && cd build cmake .. make -j 4 

5. Register the path to the OpenMP library and the corresponding headers
In order for the compiler and linker to find the components they need when building XGBoost, you need to set the paths to them. To do this, add the following lines to ~ / .bash_profile:

 export C_INCLUDE_PATH=PATH_TO_LIBOMP/libomp/exports/common/include/:$C_INCLUDE_PATH export CPLUS_INCLUDE_PATH=PATH_TO_LIBOMP/libomp/exports/common/include/:$CPLUS_INCLUDE_PATH export LIBRARY_PATH=PATH_TO_LIBOMP/libomp/exports/mac_32e/lib/:$LIBRARY_PATH export DYLD_LIBRARY_PATH=PATH_TO_LIBOMP/libomp/exports/mac_32e/lib/:$DYLD_LIBRARY_PATH 

As you can guess, PATH_TO_LIBOMP is the path to the folder where the library is located. For the changes to take effect, you must run the command

 source ~/.bash_profile 

You need to make sure that everything works correctly. To do this, create a sample program

 #include <omp.h> #include <stdio.h> #include <iostream> int main(int argc, char** argv) { std::cout << "Hello!" << std::endl; #pragma omp parallel printf("Hello from thread %d, nthreads %d\n", omp_get_thread_num(), omp_get_num_threads()); return 0; } 

and try to compile it:

 PATH_TO_CLANGOMP/clang-omp/build/bin/clang++ sample.cpp -o sample -fopenmp 

If everything is good, launching the program, we will see messages from several threads on the screen.

6. Build XGBoost
We are almost there. In the xgboost folder is the Makefile, the first lines of which need to be edited as follows:

 export CC = PATH_TO_CLANGOMP/clang-omp/build/bin/clang export CXX = PATH_TO_CLANGOMP/clang-omp/build/bin/clang++ export MPICXX = mpicxx export LDFLAGS= -pthread -lm export CFLAGS = -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -fopenmp 

We collect:

 make -j 4 

Victory.

7. Install Bundle for Python

 cd wrapper python setup.py install 

To ensure that the XGBoost binding is working properly, you can use the demo scripts in xgboost / demo.

8. Conquer Leaderboard Tops on Kaggle
The electricity suddenly went off, and this most important part, unfortunately, was lost and will be illuminated in the future.