---
metadata:
  - name: generator
    content: Diplodoc Platform v5.50.3
alternate:
  - https://catboost.ai/docs/en/installation/cli-installation-multi-node-installation.md
---
> **Documentation Index:** Fetch the complete configuration index at https://catboost.ai/docs/en/llms.txt

# Build the binary with MPI support from a local source repository (GPU only)


CatBoost provides a beta version of multi-node GPU training. Only the <q>feature parallel</q> learning scheme is currently supported. Therefore, only datasets with many features gain the benefit from multi-host multi-GPU support.

## Build

To build the command-line package with MPI support from a local copy of the CatBoost repository:

{% note warning %}

<!-- source: en/_includes/work_src/reusage-installation/ya-make-to-cmake-switch.md -->
CatBoost uses [CMake](https://cmake.org/)-based build process since [this commit](https://github.com/catboost/catboost/commit/c5c642ca0b8e093336d0229ac4b14c78db3915bb). Previously `Ya Make` (Yandex's build system) had been used.
<!-- endsource: en/_includes/work_src/reusage-installation/ya-make-to-cmake-switch.md -->

Select the appropriate build method below accordingly.

{% endnote %}

### Requirements

* Common build environment setup for [Ya Make](https://catboost.ai/docs/en/installation/build-environment-setup-for-ya-make.md) or [CMake](https://catboost.ai/docs/en/installation/build-environment-setup-for-cmake.md). 
* Some MPI implementation library (for example, `OpenMPI`) installation is required.

### Source code

<!-- source: en/_includes/work_src/reusage-installation/get-source-code-from-github.md -->
CatBoost source code is stored as a [Git](https://git-scm.com/) repository on GitHub at <https://github.com/catboost/catboost/>. You can obtain a local copy of this Git repository by running the following command from a command line interpreter (you need to have Git command line tools installed):

    git clone https://github.com/catboost/catboost.git
<!-- endsource: en/_includes/work_src/reusage-installation/get-source-code-from-github.md -->

### Build using CMake  {#cmake}

Build `catboost` target with CUDA support enabled and `-DUSE_MPI=1` flag passed to `cmake` (possibly through `build_native.py`). See [Build native artifacts](https://catboost.ai/docs/en/installation/build-native-artifacts.md).

Another flag can be also used:

  `-DWITHOUT_CUDA_AWARE_MPI=1` is required if the installed MPI library does not support CUDA-aware MPI with multiple GPUs for each MPI process on a host.

### Build using Ya Make  {#ya-make}

1. Add the following environment variables:
    ```no-highlight
    export CPLUS_INCLUDE_PATH=$PATH_TO_MPI/include:$CPLUS_INCLUDE_PATH
    export LIBRARY_PATH=$PATH_TO_MPI/lib:$LIBRARY_PATH
    export LD_LIBRARY_PATH=$PATH_TO_MPI/lib:$LD_LIBRARY_PATH
    ```

1. Open the `catboost/catboost/app` directory from the local copy of the CatBoost repository.

1. Build the command-line version of CatBoost with MPI support:

    ```
    ya make -r -DUSE_MPI -DCUDA_ROOT=<path to CUDA> [-DWITHOUT_CUDA_AWARE_MPI]
    ```

    `-DWITHOUT_CUDA_AWARE_MPI` is required if the installed MPI library does not support CUDA-aware MPI with multiple GPUs for each MPI process on a host.

## Usage

<!-- source: en/_includes/work_src/reusage-code-examples/nvidia-driver-reqs.md -->
Training or inference on CUDA-enabled GPUs requires NVIDIA Driver of version 450.80.02 or higher.
<!-- endsource: en/_includes/work_src/reusage-code-examples/nvidia-driver-reqs.md -->

Refer to the MPI library documentation for guides on running MPI jobs on multiple hosts.

{% note info %}

Performance notes:
- CatBoost requires one working thread per GPU, one thread for routing network messages, and several threads for preprocessing data. Use an appropriate CPU binding policy. It is recommended to bind the application to the full host or PCI root complex used by CUDA devices for optimal performance. See the MPI documentation for more details.
- A fast network connection (like InfiniBand) is required in order to gain the benefit from multi-node support. Slow networks with network capacity of 1Gb/s or less can whittle down the advantages of multi-node training.

{% endnote %}