# Classification: objectives and metrics

## Objectives and metrics

### Logloss

$\displaystyle\frac{ - \sum\limits_{i=1}^N w_{i}\left(c_i \log(p_{i}) + (1-c_{i}) \log(1 - p_{i})\right)}{\sum\limits_{i = 1}^{N} w_{i}}$

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### CrossEntropy

$\displaystyle\frac{- \sum\limits_{i=1}^N w_{i} \left(t_{i} \log(p_{i}) + (1 - t_{i}) \log(1 - p_{i})\right)}{\sum\limits_{i = 1}^{N} w_{i}}$

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### Precision

$\frac{TP}{TP + FP}$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### Recall

$\frac{TP}{TP+FN}$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### F

$(1 + \beta^2) \cdot \frac{Precision * Recall}{(\beta^2 \cdot Precision) + Recall}$

Can't be used for optimization. See more.

User-defined parameters

beta

The $\beta$ parameter of the F metric.

Valid values are real numbers in the following range: $(0; +\infty)$.

Default: This parameter is obligatory (the default value is not defined)

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### F1

$2 \frac{Precision * Recall}{Precision + Recall}$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### BalancedAccuracy

$\frac{1}{2} \left(\frac{TP}{P} + \frac{TN}{N} \right)$
User-defined parameters:

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### BalancedErrorRate

$\frac{1}{2} \left( \displaystyle\frac{FP}{TN + FP} + \displaystyle\frac{FN}{FN + TP} \right)$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### MCC

$\displaystyle\frac{TP * TN - FP * FN}{\sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}}$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### Accuracy

$\frac{TP + TN}{\sum\limits_{i=1}^{N} w_{i}}$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### CtrFactor

$\displaystyle\frac{\left(\sum\limits_{i = 1}^{N} w_{i} t_{i}/N\right)}{\left(\sum\limits_{i = 1}^{N} w_{i} p_{i} /N\right)}$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### AUC

The calculation of this metric is disabled by default for the training dataset to speed up the training. Use the hints=skip_train~false parameter to enable the calculation.

#### Classic

$\displaystyle\frac{\sum I(a_{i}, a_{j}) \cdot w_{i} \cdot w_{j}} {\sum w_{i} \cdot w_{j}}$
The sum is calculated on all pairs of objects $(i,j)$ such that:

• $t_{i} = 0$
• $t_{j} = 1$
• $I(x, y) = \begin{cases} 0 { , } & x < y \\ 0.5 { , } & x=y \\ 1 { , } & x>y \end{cases}$

Refer to the Wikipedia article for details.

If the target type is not binary, then every object with target value $t$ and weight $w$ is replaced with two objects for the metric calculation:

• $o_{1}$ with weight $t \cdot w$ and target value 1
• $o_{2}$ with weight $(1 – t) \cdot w$ and target value 0.

Target values must be in the range [0; 1].

#### Ranking

$\displaystyle\frac{\sum I(a_{i}, a_{j}) \cdot w_{i} \cdot w_{j}} {\sum w_{i} * w_{j}}$

The sum is calculated on all pairs of objects $(i,j)$ such that:

• $t_{i} < t_{j}$
• $I(x, y) = \begin{cases} 0 { , } & x < y \\ 0.5 { , } & x=y \\ 1 { , } & x>y \end{cases}$

User-defined parameters

type

The type of AUC. Defines the metric calculation principles.

Default: Ranking.
Possible values: Classic, Ranking.
Examples: AUC:type=Classic, AUC:type=Ranking.

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: False.
Examples: QueryAUC:type=Ranking;use_weights=False.

### QueryAUC

#### Classic type

$\displaystyle\frac{ \sum_q \sum_{i, j \in q} \sum I(a_{i}, a_{j}) \cdot w_{i} \cdot w_{j}} { \sum_q \sum_{i, j \in q} \sum w_{i} \cdot w_{j}}$
The sum is calculated on all pairs of objects $(i,j)$ such that:

• $t_{i} = 0$
• $t_{j} = 1$
• $I(x, y) = \begin{cases} 0 { , } & x < y \\ 0.5 { , } & x=y \\ 1 { , } & x>y \end{cases}$

Refer to the Wikipedia article for details.

If the target type is not binary, then every object with target value $t$ and weight $w$ is replaced with two objects for the metric calculation:

• $o_{1}$ with weight $t \cdot w$ and target value 1
• $o_{2}$ with weight $(1 – t) \cdot w$ and target value 0.

Target values must be in the range [0; 1].

#### Ranking type

$\displaystyle\frac{ \sum_q \sum_{i, j \in q} \sum I(a_{i}, a_{j}) \cdot w_{i} \cdot w_{j}} { \sum_q \sum_{i, j \in q} \sum w_{i} * w_{j}}$

The sum is calculated on all pairs of objects $(i,j)$ such that:

• $t_{i} < t_{j}$
• $I(x, y) = \begin{cases} 0 { , } & x < y \\ 0.5 { , } & x=y \\ 1 { , } & x>y \end{cases}$

Can't be used for optimization. See more.

User-defined parameters

type

The type of QueryAUC. Defines the metric calculation principles.

Default: Ranking.
Possible values: Classic, Ranking.
Examples: QueryAUC:type=Classic, QueryAUC:type=Ranking.

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: False.
Examples: QueryAUC:type=Ranking;use_weights=False.

### NormalizedGini

See AUC.

$2 AUC - 1$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### BrierScore

$\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i}\left(p_{i} - t_{i} \right)^{2}}{\sum\limits_{i=1}^{N} w_{i}}$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### HingeLoss

$\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} max\{1 - t_{i} p_{i}, 0\}}{\sum\limits_{i=1}^{N} w_{i}} , t_{i} = \pm 1$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### HammingLoss

$\displaystyle\frac{\sum\limits_{i = 1}^{N} w_{i} [[p_{i} > 0.5] == t_{i}]]}{\sum\limits_{i=1}^{N} w_{i}}$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### ZeroOneLoss

$1 - Accuracy$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### Kappa

$1 - \displaystyle\frac{1 - Accuracy}{1 - RAccuracy}$

$RAccuracy = \displaystyle\frac{(TN + FP) (TN + FN) + (FN + TP) (FP + TP)}{(\sum\limits_{i=1}^{N} w_{i})^{2}}$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### WKappa

See the formula on page 3 of the A note on the linearly weighted kappa coefficient for ordinal scales paper.

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

### LogLikelihoodOfPrediction

The calculation consists of the following steps:

1. Define the sum of weights ($W$) and the mean target ($\bar{t}$):

$W = \sum\limits_{i} w_{i}$

$\bar{t} = \frac{1}{W} \sum\limits_{i} t_{i} w_{i}$

2. Denote log-likelihood of a constant prediction:

$ll_0 = \sum\limits_{i} w_{i} (\bar{t} \cdot log(\bar{t}) + (1 - \bar{t}) \cdot log(1 - \bar{t}))$

3. Calculate LogLikelihoodOfPrediction ($llp$), which reflects how the likelihood ($ll$) differs from the constant prediction:

$llp = \displaystyle\frac{ll(t, w) - ll_0}{\sum\limits_{i} t_{i} w_{i}}$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

## Used for optimization

Name Optimization GPU Support
Logloss + +
CrossEntropy + +
Precision - +
Recall - +
F - -
F1 - +
BalancedAccuracy - -
BalancedErrorRate - -
MCC - +
Accuracy - +
CtrFactor - -
AUC - -
QueryAUC - -
NormalizedGini - -
BrierScore - -
HingeLoss - -
HammingLoss - -
ZeroOneLoss - +
Kappa - -
WKappa - -
LogLikelihoodOfPrediction - -