MultiLabel Classification: objectives and metrics

Objectives and metrics

MultiLogloss

j=0M1i=1Nwi(cijlogpij+(1cij)log(1pij))Mi=1Nwi,\displaystyle\frac{-\sum\limits_{j=0}^{M-1} \sum\limits_{i=1}^{N} w_{i} (c_{ij} \log p_{ij} + (1-c_{ij}) \log (1 - p_{ij}) )}{M\sum\limits_{i=1}^{N}w_{i}} { ,}

where pij=σ(aij)=eaij1+eaijp_{ij} = \sigma(a_{ij}) = \frac{e^{a_{ij}}}{1 + e^{a_{ij}}} and cij0,1c_{ij} \in {0, 1}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

MultiCrossEntropy

j=0M1i=1Nwi(tijlogpij+(1tij)log(1pij))Mi=1Nwi,\displaystyle\frac{-\sum\limits_{j=0}^{M-1} \sum\limits_{i=1}^{N} w_{i} (t_{ij} \log p_{ij} + (1-t_{ij}) \log (1 - p_{ij}) )}{M\sum\limits_{i=1}^{N}w_{i}} { ,}

where pij=σ(aij)=eaij1+eaijp_{ij} = \sigma(a_{ij}) = \frac{e^{a_{ij}}}{1 + e^{a_{ij}}} and tij[0,1]t_{ij} \in [0, 1]

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

Precision

This function is calculated separately for each class k numbered from 0 to M – 1.

TPTP+FP\frac{TP}{TP + FP}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

Recall

This function is calculated separately for each class k numbered from 0 to M – 1.

TPTP+FN\frac{TP}{TP+FN}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

F1

This function is calculated separately for each class k numbered from 0 to M – 1.

2PrecisionRecallPrecision+Recall2 \frac{Precision * Recall}{Precision + Recall}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

Accuracy

The formula depends on the value of the typetype parameter:

Classic

i=1Nwij=0M1[[pij>0.5]==tij]i=1Nwi,\displaystyle\frac{\sum\limits_{i=1}^{N}w_{i} \prod\limits_{j=0}^{M-1} [[p_{ij} > 0.5]==t_{ij}]}{\sum\limits_{i=1}^{N}w_{i}} { , }

where pij=σ(aij)=eaij1+eaijp_{ij} = \sigma(a_{ij}) = \frac{e^{a_{ij}}}{1 + e^{a_{ij}}}

PerClass

This function is calculated separately for each class k numbered from 0 to M – 1.

TP+TNi=1Nwi\frac{TP + TN}{\sum\limits_{i=1}^{N} w_{i}}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

type

The type of calculated accuracy.

Possible values:

  • Classic
  • PerClass

Default: Classic

HammingLoss

j=0M1i=1Nwi[[pij>0.5]==tij]]Mi=1Nwi\displaystyle\frac{\sum\limits_{j=0}^{M-1} \sum\limits_{i = 1}^{N} w_{i} [[p_{ij} > 0.5] == t_{ij}]]}{M \sum\limits_{i=1}^{N} w_{i}}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

Used for optimization

Name Optimization
MultiLogloss +
MultiCrossEntropy +
Precision -
Recall -
F1 -
Accuracy -
HammingLoss -