MultiLabel Classification: objectives and metrics

Objectives and metrics

MultiLogloss

j=0M1i=1Nwi(cijlogpij+(1cij)log(1pij))Mi=1Nwi,\displaystyle\frac{-\sum\limits_{j=0}^{M-1} \sum\limits_{i=1}^{N} w_{i} (c_{ij} \log p_{ij} + (1-c_{ij}) \log (1 - p_{ij}) )}{M\sum\limits_{i=1}^{N}w_{i}} { ,}

where pij=σ(aij)=eaij1+eaijp_{ij} = \sigma(a_{ij}) = \frac{e^{a_{ij}}}{1 + e^{a_{ij}}} and cij0,1c_{ij} \in {0, 1}

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

MultiCrossEntropy

j=0M1i=1Nwi(tijlogpij+(1tij)log(1pij))Mi=1Nwi,\displaystyle\frac{-\sum\limits_{j=0}^{M-1} \sum\limits_{i=1}^{N} w_{i} (t_{ij} \log p_{ij} + (1-t_{ij}) \log (1 - p_{ij}) )}{M\sum\limits_{i=1}^{N}w_{i}} { ,}

where pij=σ(aij)=eaij1+eaijp_{ij} = \sigma(a_{ij}) = \frac{e^{a_{ij}}}{1 + e^{a_{ij}}} and tij[0,1]t_{ij} \in [0, 1]

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

Precision

This function is calculated separately for each class k numbered from 0 to M – 1.

TPTP+FP\frac{TP}{TP + FP}

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

Recall

This function is calculated separately for each class k numbered from 0 to M – 1.

TPTP+FN\frac{TP}{TP+FN}

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

F

This function is calculated separately for each class k numbered from 0 to M – 1.

(1+β2)PrecisionRecall(β2Precision)+Recall(1 + \beta^2) \cdot \frac{Precision * Recall}{(\beta^2 \cdot Precision) + Recall}

Can't be used for optimization. See more.

User-defined parameters

beta

The β\beta parameter of the F metric.

Valid values are real numbers in the following range: (0;+)(0; +\infty).

Default: This parameter is obligatory (the default value is not defined)

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

F1

This function is calculated separately for each class k numbered from 0 to M – 1.

2PrecisionRecallPrecision+Recall2 \frac{Precision * Recall}{Precision + Recall}

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

Accuracy

The formula depends on the value of the typetype parameter:

Classic

i=1Nwij=0M1[[pij>0.5]==tij]i=1Nwi,\displaystyle\frac{\sum\limits_{i=1}^{N}w_{i} \prod\limits_{j=0}^{M-1} [[p_{ij} > 0.5]==t_{ij}]}{\sum\limits_{i=1}^{N}w_{i}} { , }

where pij=σ(aij)=eaij1+eaijp_{ij} = \sigma(a_{ij}) = \frac{e^{a_{ij}}}{1 + e^{a_{ij}}}

PerClass

This function is calculated separately for each class k numbered from 0 to M – 1.

TP+TNi=1Nwi\frac{TP + TN}{\sum\limits_{i=1}^{N} w_{i}}

Can't be used for optimization. See more.

User-defined parameters

type

The type of calculated accuracy.

Default: Classic.
Possible values: Classic, PerClass.

HammingLoss

j=0M1i=1Nwi[[pij>0.5]==tij]]Mi=1Nwi\displaystyle\frac{\sum\limits_{j=0}^{M-1} \sum\limits_{i = 1}^{N} w_{i} [[p_{ij} > 0.5] == t_{ij}]]}{M \sum\limits_{i=1}^{N} w_{i}}

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

Used for optimization

Name Optimization GPU Support
MultiLogloss + -
MultiCrossEntropy + -
Precision - -
Recall - -
F - -
F1 - -
Accuracy - -
HammingLoss - -