---
metadata:
  - name: generator
    content: Diplodoc Platform v5.50.3
alternate:
  - https://catboost.ai/docs/en/concepts/loss-functions-multiclassification.md
---
> **Documentation Index:** Fetch the complete configuration index at https://catboost.ai/docs/en/llms.txt

# Multiclassification: objectives and metrics

- [Objectives and metrics](#objectives-and-metrics)
- [Used for optimization](#usage-information)

## Objectives and metrics

### MultiClass {#MultiClass}

$\displaystyle\frac{\sum\limits_{i=1}^{N}w_{i}\log\left(\displaystyle\frac{e^{a_{it_{i}}}}{ \sum\limits_{j=0}^{M - 1}e^{a_{ij}}} \right)}{\sum\limits_{i=1}^{N}w_{i}} { ,}$

$t \in \{0, ..., M - 1\}$

**Usage information**  See [more](#usage-information).

**User-defined parameters**

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->
{% cut "use_weights" %}

Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.

_Default:_ true

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->


### MultiClassOneVsAll {#MultiClassOneVsAll}

$\displaystyle\frac{\frac{1}{M}\sum\limits_{i = 1}^N w_i \sum\limits_{j = 0}^{M - 1} [j = t_i] \log(p_{ij}) + [j \neq t_i] \log(1 - p_{ij})}{\sum\limits_{i = 1}^N w_i} { ,}$

$t \in \{0, ..., M - 1\}$

**Usage information**  See [more](#usage-information).

**User-defined parameters**

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->
{% cut "use_weights" %}

Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.

_Default:_ true

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->


### Precision {#Precision}

<!-- source: en/_includes/work_src/reusage/loss-function__for-multiclass.md -->
This function is calculated separately for each class k numbered from 0 to M – 1.
<!-- endsource: en/_includes/work_src/reusage/loss-function__for-multiclass.md -->

$\frac{TP}{TP + FP}$

**Can't be used for optimization.**  See [more](#usage-information).

**User-defined parameters**

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->
{% cut "use_weights" %}

Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.

_Default:_ true

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->


### Recall {#Recall}

<!-- source: en/_includes/work_src/reusage/loss-function__for-multiclass.md -->
This function is calculated separately for each class k numbered from 0 to M – 1.
<!-- endsource: en/_includes/work_src/reusage/loss-function__for-multiclass.md -->

$\frac{TP}{TP+FN}$

**Can't be used for optimization.**  See [more](#usage-information).

**User-defined parameters**

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->
{% cut "use_weights" %}

Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.

_Default:_ true

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->

### F {#F}

<!-- source: en/_includes/work_src/reusage/loss-function__for-multiclass.md -->
This function is calculated separately for each class k numbered from 0 to M – 1.
<!-- endsource: en/_includes/work_src/reusage/loss-function__for-multiclass.md -->

$(1 + \beta^2) \cdot  \frac{Precision * Recall}{(\beta^2 \cdot Precision) + Recall}$

**Can't be used for optimization.**  See [more](#usage-information).

**User-defined parameters**

<!-- source: en/_includes/work_src/reusage-loss-functions/beta_for_F__desc.md -->
{% cut "beta" %}

The $\beta$ parameter of the F metric.

Valid values are real numbers in the following range:  $(0; +\infty)$.

_Default:_ This parameter is obligatory (the default value is not defined)

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/beta_for_F__desc.md -->

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->
{% cut "use_weights" %}

Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.

_Default:_ true

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->

### F1 {#F1}

<!-- source: en/_includes/work_src/reusage/loss-function__for-multiclass.md -->
This function is calculated separately for each class k numbered from 0 to M – 1.
<!-- endsource: en/_includes/work_src/reusage/loss-function__for-multiclass.md -->

$2 \frac{Precision * Recall}{Precision + Recall}$

**Can't be used for optimization.**  See [more](#usage-information).

**User-defined parameters**

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->
{% cut "use_weights" %}

Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.

_Default:_ true

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->

### TotalF1 {#totalF1}

The formula depends on the value of the average parameter:

#### Weighted

$\frac{\sum\limits_{i=1}^{M} w_{i} F1_{i}}{\sum\limits_{i=1}^{M}w_{i}} {, where}$

$w_{i}$ is the sum of the weights of the documents which correspond to the i-th class. If document weights are not specified $w_{i}$ stands for the number of times the i-th class is found among the label values.

#### Macro

$\displaystyle\frac{\sum\limits_{i=1}^{M}F1_{i}}{M}$

#### Micro

$TotalF1 = \displaystyle\frac{2 \cdot TP}{2 \cdot TP + FP + FN} {, where}$
- $TP = \sum\limits_{i=1}^{M} TP_{i}$
- $FP = \sum\limits_{i=1}^{M} FP_{i}$
- $FN = \sum\limits_{i=1}^{M} FN_{i}$

**Can't be used for optimization.**  See [more](#usage-information).

**User-defined parameters**

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->
{% cut "use_weights" %}

Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.

_Default:_ true

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->

{% cut "average" %}

The method for averaging the value of the metric that is initially individually calculated for each class.

_Default:_  `Weighted`.
_Possible values:_ `Weighted`, `Macro`, `Micro`.

{% endcut %}

### MCC {#MCC}

This functions is defined in terms of a $k \times k$ confusion matrix $C$ (where k is the number of classes):

$\displaystyle\frac{\sum\limits_{k}\sum\limits_{l}\sum\limits_{m} C_{kk} C_{lm} - C_{kl}C_{mk}}{\sqrt{\sum\limits_{k} \left(\sum\limits_{l} C_{kl}\right) \left(\sum\limits_{k' | k' \neq k} \sum\limits_{l'} C_{k'l'}\right)}\sqrt{\sum\limits_{k} \left(\sum\limits_{l} C_{lk}\right) \left(\sum\limits_{k' | k' \neq k} \sum\limits_{l'} C_{l' k'}\right)}}$

See the [Wikipedia article](https://en.wikipedia.org/wiki/Matthews_correlation_coefficient) for more details.

**Can't be used for optimization.**  See [more](#usage-information).

**User-defined parameters**

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->
{% cut "use_weights" %}

Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.

_Default:_ true

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->

### Accuracy {#Accuracy}

$\displaystyle\frac{\sum\limits_{i=1}^{N}w_{i}[argmax_{j=0,...,M - 1}(a_{ij})==t_{i}]}{\sum\limits_{i=1}^{N}w_{i}} { , }$

$t \in \{0, ..., M - 1\}$

**Can't be used for optimization.**  See [more](#usage-information).

**User-defined parameters**

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->
{% cut "use_weights" %}

Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.

_Default:_ true

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->

### HingeLoss {#HingeLoss}

See the [Wikipedia article](https://en.wikipedia.org/wiki/Hinge_loss).

**Can't be used for optimization.**  See [more](#usage-information).

**User-defined parameters**

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->
{% cut "use_weights" %}

Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.

_Default:_ true

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->

### HammingLoss {#HammingLoss}

$\displaystyle\frac{\sum\limits_{i = 1}^{N} w_{i} [argmax_{j=0,...,M - 1}(a_{ij})\neq t_{i}]}{\sum\limits_{i = 1}^{N} w_{i}}$

**Can't be used for optimization.**  See [more](#usage-information).

**User-defined parameters**

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->
{% cut "use_weights" %}

Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.

_Default:_ true

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->

### ZeroOneLoss {#ZeroOneLoss}

$1 - Accuracy$

**Can't be used for optimization.**  See [more](#usage-information).

**User-defined parameters**

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->
{% cut "use_weights" %}

Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.

_Default:_ true

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->

### Kappa {#Kappa}

$1 - \displaystyle\frac{1 - Accuracy}{1 - RAccuracy}$

$RAccuracy = \displaystyle\frac{\sum\limits_{k=0}^{M - 1} n_{k_{a}}n_{k_{t}}}{(\sum\limits_{i=1}^{N}w_{i})^{2}}$

$k_{a}$ is the weighted number of times class k is predicted by the model

$k_{t}$ is the weighted number of times class k is set as the label for input objects

**Can't be used for optimization.**  See [more](#usage-information).

**User-defined parameters**

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->
{% cut "use_weights" %}

Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.

_Default:_ true

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->

### WKappa {#WKappa}

See the formula on page 3 of the [A note on the linearly weighted kappa coefficient for ordinal scales](https://orbi.uliege.be/bitstream/2268/2262/1/STATMED-174.pdf) paper.

**Can't be used for optimization.**  See [more](#usage-information).

**User-defined parameters**

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->
{% cut "use_weights" %}

Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.

_Default:_ true

{% endcut %}
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__with__default__value.md -->


### AUC {#AUC}

The calculation of this metric is disabled by default for the training dataset to speed up the training. Use the `hints=skip_train~false` parameter to enable the calculation.

- Mu

    Refer to the [A Performance Metric for Multi-Class Machine Learning Models](http://proceedings.mlr.press/v97/kleiman19a/kleiman19a.pdf) paper for calculation principles

- OneVsAll

    The value is calculated separately for each class k numbered from 0 to M–1 according to the [binary classification calculation principles](https://catboost.ai/docs/en/concepts/loss-functions-classification.md#auc__full-desc). The objects of class k are considered positive, while all others are considered negative.

**Can't be used for optimization.**  See [more](#usage-information).

**User-defined parameters**

{% cut "use_weights" %}

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__without__note.md -->
Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__without__note.md -->

_Default_: `False`.
_Examples_: `AUC:type=Ranking;use_weights=False`.

{% endcut %}

{% cut "type" %}

<!-- source: en/_includes/work_src/reusage-loss-functions/type_of_auc__p.md -->
The type of AUC. Defines the metric calculation principles.
<!-- endsource: en/_includes/work_src/reusage-loss-functions/type_of_auc__p.md -->

_Default:_ Mu
_Possible values:_ `Mu`, `OneVsAll`.
_Examples_: `AUC:type=Mu`, `AUC:type=OneVsAll`.

{% endcut %}

{% cut "misclass_cost_matrix" %}

The matrix _M_ with misclassification cost values. $M[i,j]$ in this matrix is the cost of classifying an object as a member of the class _i_ when its' actual class is _j_. Applicable only if the used type of AUC is Mu.

Format for a matrix of size C:
```
<Value for M[0,0]>, <Value for M[0,1]>, ..., <Value for M[0,C-1]>, <Value for M[1,0]>, ..., <Value for M[C-1,0]>, ..., <Value for M[C-1,C-1]>
```

All diagonal elements $M[i, j]$ (such that _i=j_) must be equal to 0.

{% note info %}

The `type` parameter is optional and is assumed to be set to `Mu` if the parameter is explicitly specified.

{% endnote %}

_Default:_  All non-diagonal matrix elements are set to 1. All diagonal elements $М[i, j]$ (such that _i = j_) are set to 0.
_Examples:_ Three classes — `AUC:misclass_cost_matrix=0/0.5/2/1/0/1/0/0.5/0`, Two classes — `AUC:type=Mu;misclass_cost_matrix=0/0.5/1/0`.

{% endcut %}

<!-- source: en/_includes/concepts/pr-auc.md -->
### PRAUC {#PRAUC}

PRAUC is the area under the curve $Precision(q)$ vs $Recall(q)$ for $q \in [0,1]$ where $Precision(q)$ and $Recall(q)$ are defined as follows.
$$Precision(q) = \frac{TP(q)}{TP(q) + FP(q)}, Recall(q) = \frac{TP(q)}{TP(q) + FN(q)}$$

Above $TP(q)$, $FP(q)$, $FN(q)$ are weights of the true positive, false positive, and false negative samples, respectively.

To calculate PRAUC for a binary classification model, specify type `Classic`.
In this case, $TP(q)=\sum w_i [p_i > q] c_i$, etc.

To calculate PRAUC for a multi-classification model, specify type `OneVsAll`.
In this case, positive samples are samples having class 0, all other samples are negative, and $TP(q)=\sum w_i [p_{i0} > q] [c_i = 0]$, etc.

{% cut "type" %}

The type of PRAUC. Defines the metric calculation principles.

Type `Classic` is compatible with binary classification models.
Type `OneVsAll` is compatible with multi-classification models.

_Default_: `Classic`.
_Possible values_: `Classic`, `OneVsAll`.
_Examples_: `PRAUC:type=Classic`, `PRAUC:type=OneVsAll`.

{% endcut %}

{% cut "use_weights" %}

<!-- source: en/_includes/work_src/reusage-loss-functions/use-weights__desc__without__note.md -->
Use object/group weights to calculate metrics if the specified value is <q>true</q> and set all weights to <q>1</q> regardless of the input data if the specified value is <q>false</q>.
<!-- endsource: en/_includes/work_src/reusage-loss-functions/use-weights__desc__without__note.md -->

_Default_: `False`.
_Examples_: `PRAUC:type=Classic;use_weights=False`.

{% endcut %}
<!-- endsource: en/_includes/concepts/pr-auc.md -->


## Used for optimization {#usage-information}

| Name                                                          | Optimization            | GPU Support             |
----------------------------------------------------------------|-------------------------|-------------------------|
[MultiClass](#MultiClass)                 |     +                   |     +                   |
[MultiClassOneVsAll](#MultiClassOneVsAll) |     +                   |     +                   |
[Precision](#Precision)                   |     -                   |     +                   |
[Recall](#Recall)                         |     -                   |     +                   |
[F](#F)                                   |     -                   |     -                   |
[F1](#F1)                                 |     -                   |     +                   |
[TotalF1](#totalF1)                       |     -                   |     +                   |
[MCC](#MCC)                               |     -                   |     +                   |
[Accuracy](#Accuracy)                     |     -                   |     +                   |
[HingeLoss](#HingeLoss)                   |     -                   |     -                   |
[HammingLoss](#HammingLoss)               |     -                   |     -                   |
[ZeroOneLoss](#ZeroOneLoss)               |     -                   |     +                   |
[Kappa](#Kappa)                           |     -                   |     -                   |
[WKappa](#WKappa)                         |     -                   |     -                   |
[AUC](#AUC)                               |     -                   |     -                   |