Build Python package from source on Linux and macOS using Ya Make

Note

CatBoost uses CMake-based build process since this commit. Previously Ya Make (Yandex's build system) had been used.

This page describes building using Ya Make. For building using CMake see documentation here.

Dependencies and requirements

  1. Setup environment for ya make build

  2. The following packages are required for installation:

  • python3
  • python3-dev
  • numpy
  • pandas

Training or inference on CUDA-enabled GPUs requires NVIDIA Driver of version 450.80.02 or higher.

Building steps

To build the Python package from source on Linux and macOS:

  1. Clone the repository:

    git clone https://github.com/catboost/catboost.git

  2. Open the catboost/catboost/python-package/catboost directory from the local copy of the CatBoost repository.

  3. Compile the packages using one of the following methods:

    • Use one of the Python versions provided by the ya make utility:

      ../../../ya make -r -DUSE_ARCADIA_PYTHON=no -DUSE_SYSTEM_PYTHON=<Python version> [optional parameters]

    • Use one of the Python versions installed on the machine:

      ../../../ya make -r -DUSE_ARCADIA_PYTHON=no -DOS_SDK=local -DPYTHON_CONFIG=<path to the required python-config> [optional parameters]

    Parameter Description
    Parameters that define the Python version to use for compiling. Only one of the following blocks of options can be used at a time
    Use one of the Python versions provided by the ya make utility
    -DUSE_SYSTEM_PYTHONThe version of Python to use for compiling the package on machines without installed Python.

    The following Python versions are supported and can be defined as values for this parameter:

    • 2.7
    • 3.4
    • 3.5
    • 3.6
    Use one of the Python versions installed on the machine
    -DPYTHON_CONFIGDefines the path to the configuration of an installed Python version to use for compiling the package.

    Value examples:

    • python2-config for Python 2
    • python3-config for Python 3
    • /usr/bin/python2.7-config
    • The configuration must be explicitly named python3-config to successfully build the package for Python 3.
    • Manually redefine the following variables when encountering problems with the Python configuration:
      • -DPYTHON_INCLUDE
      • -DPYTHON_LIBRARIES
      • -DPYTHON_LDFLAGS
      • -DPYTHON_FLAGS
      • -DPYTHON_BIN
    Optional parameters
    -DCUDA_ROOTThe path to CUDA. This parameter is required to support training on GPU.
    -DHAVE_CUDA=noDisable CUDA support. This speeds up compilation.

    By default, the package is built with CUDA support if CUDA Toolkit is installed.
    -oThe directory to output the compiled package to. By default, the current directory is used.

    For example, the following command builds the package for Python 3 with training on GPU support:

    ../../../ya make -r -DUSE_ARCADIA_PYTHON=no -DOS_SDK=local -DPYTHON_CONFIG=python3-config -DCUDA_ROOT=/usr/local/cuda

    Note

    The required version of Xcode for building on macOS is specified on the NVIDIA site when downloading the CUDA toolkit.

  4. Add the current directory to PYTHONPATH:

    cd ../; export PYTHONPATH=$PYTHONPATH:$(pwd)

Troubleshooting

Error type Message format Troubleshooting tips
AttributeError type object '_catboost.<something>' has no attribute 'reduce_cython'
Example:
AttributeError: type object '_catboost._FloatArrayWrapper' has no attribute 'reduce_cython'
AttributeError: type object '_catboost.Py_ITypedSequencePtr' has no attribute 'reduce_cython'

Install or update the following packages:
- numpy
- pandas
Previous
Build from source on Linux and macOS
Next
Build from source on Windows