Build Python package from source

Warning

This page describes the new approach for building using standard Python tools that works since this commit.

For building with earlier versions see these pages:

Source code

CatBoost source code is stored as a Git repository on GitHub at https://github.com/catboost/catboost/. You can obtain a local copy of this Git repository by running the following command from a command line interpreter (you need to have Git command line tools installed):

git clone https://github.com/catboost/catboost.git

Later in this document $CATBOOST_SRC_ROOT refers to the root dir of the local working copy of the source code cloned from the GitHub CatBoost repository.

Dependencies and requirements

  1. As CatBoost Python package has a native extension library as its' core build environment setup for CMake is required.

  2. Other setup dependencies that can be formulated as python packages are listed in pyproject.toml's build-system.requires and in setup.py in standard setup_requires parameter and processed using standard Python tools.

    Note

    For some reason Python 3.12 fails to automatically resolve build/setup dependencies in a way that they are buildable so it is recommended to install the following packages using pip explicitly:
    - setuptools
    - wheel
    - jupyterlab (3.x, 4.x is not supported yet, see the relevant issue)
    - conan (1.62.0+, but 1.x series, 2.x is not supported yet, see the relevant issue)

  3. For building CatBoost visualization widget bundled together with the python package (enabled by default) additional setup is required:

    1. Node.js installation with npm command accessible from the shell.
    2. rimraf Node.js package installed with npm's --global option (this way rimraf command will be accessible from the shell).
    3. yarn package manager, version from 1.x series, 1.22.10 or later. Installed with npm's --global option (this way yarn command will be accessible from the shell)
      An example command to install: npm install --global yarn@1.22.10.

    If you don't need CatBoost visualization widget support you can disable it's building and bundling with the CatBoost python package by passing --no-widget build/installation option.

  4. Installation dependencies are listed in setup.py in standard install_requires parameter and processed using standard Python tools.

Training or inference on CUDA-enabled GPU requires NVIDIA Driver of version 450.80.02 or higher.

Building

Open the $CATBOOST_SRC_ROOT/catboost/python-package directory from the local copy of the CatBoost repository.

Use Python's standard procedures:

Build the wheel distribution

Warning

Note that built Python wheels will be binary compatible only with the same Python X.Y versions.

python setup.py bdist_wheel <options>

Options can be listed by calling python setup.py bdist_wheel --help.

One important option is --prebuilt-extensions-build-root-dir=<path>. It allows to use already built binary _catboost extension shared library. See Build native artifacts. Set this option value to $CMAKE_BINARY_DIR.

The resulting wheel distribution will be created in dist/catboost-<version>-<...>.whl

Build the source distribution (sdist)

python -m build --sdist

The resulting source distribution will be created in dist/catboost-<version>.tar.gz file.

Other useful commands

  • build_widget. Build CatBoost widget.

    python setup.py build_widget
    

    Useful if widget code remains unchanged but you want to rebuild other parts. Then run build_widget once and then in subsequent calls to bdist_wheel or other commands use --prebuilt-widget option.

Installation

Directly from the source directory

Builds in the process. So build environment setup for CMake is required.

python -m pip install . <options>

You can pass options to setup.py's install stage by using --install-option options like this:

python -m pip install . --install-option=--with-hnsw --install-option=--with-cuda=/usr/local/cuda-11

Create editable install

Builds in the process. So build environment setup for CMake is required.

python -m pip install --editable . <options>

You can pass options to setup.py's install stage by using --install-option options like this:

python -m pip install  --install-option=--with-hnsw --install-option=--with-cuda=/usr/local/cuda-11/ --editable .

Install from the built wheel

python -m pip install <path-to-wheel>

Install from the source distribution

Builds in the process. So build environment setup for CMake is required.

python -m pip install <path-to-sdist-tar.gz>