Regression: objectives and metrics

Objectives and metrics

MAE

i=1Nwiaitii=1Nwi\frac{\sum\limits_{i=1}^{N} w_{i} | a_{i} - t_{i}| }{\sum\limits_{i=1}^{N} w_{i}}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

MAPE

i=1NwiaitiMax(1,ti)i=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} \displaystyle\frac{|a_{i}- t_{i}|}{Max(1, |t_{i}|)}}{\sum\limits_{i=1}^{N}w_{i}}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

Poisson

i=1Nwi(eaiaiti)i=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} \left(e^{a_{i}} - a_{i}t_{i}\right)}{\sum\limits_{i=1}^{N}w_{i}}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

Quantile

i=1N(α1(tiai))(tiai)wii=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N} (\alpha - 1(t_{i} \leq a_{i}))(t_{i} - a_{i}) w_{i} }{\sum\limits_{i=1}^{N} w_{i}}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

alpha

The coefficient used in quantile-based losses.

Default: 0.5

RMSE

i=1N(aiti)2wii=1Nwi\displaystyle\sqrt{\displaystyle\frac{\sum\limits_{i=1}^N (a_{i}-t_{i})^2 w_{i}}{\sum\limits_{i=1}^{N}w_{i}}}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

RMSEWithUncertainty

1Ni=1Nlogp(tiai)=1Ni=1Nlog(12πσ2exp((yμ)22σ2))=C+1Ni=1N(ai,1+12exp(2ai,1(tiai,0)2))-\frac{1}{N} \sum_{i=1}^N \log p(t_i \vert a_i) = -\frac{1}{N} \sum_{i=1}^N \log(\frac{1}{2 \pi\sigma^2} \exp(-\frac{(y-\mu)^2}{2\sigma^2})) = C +\frac{1}{N}\sum_{i=1}^N \left(a_{i,1} + \frac{1}{2} \exp(-2 a_{i,1} (t_i - a_{i, 0})^2) \right), where t is target, a 2-dimensional approx a0a_0 is target predict, a1a_1 is logσ\log \sigma predict, and pp has normal distribution p(ta)=N(ya0,e2a1)=N(yμ,σ2)=12πσ2exp((yμ)22σ2)p(t \vert a) = N(y \vert a_0, e^{2a_1}) = N(y \vert \mu, \sigma^2) = \frac{1}{2 \pi\sigma^2} \exp(-\frac{(y-\mu)^2}{2\sigma^2})

See the Uncertainty section for more details.

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

LogLinQuantile

Depends on the condition for the ratio of the label value and the resulting value:
{i=1Nαtieaiwii=1Nwiti>eaii=1N(1α)tieaiwii=1Nwitieai\begin{cases} \displaystyle\frac{\sum\limits_{i=1}^{N} \alpha |t_{i} - e^{a_{i}} | w_{i}}{\sum\limits_{i=1}^{N} w_{i}} & t_{i} > e^{a_{i}} \\ \displaystyle\frac{\sum\limits_{i=1}^{N} (1 - \alpha) |t_{i} - e^{a_{i}} | w_{i}}{\sum\limits_{i=1}^{N} w_{i}} & t_{i} \leq e^{a_{i}} \end{cases}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

alpha

The coefficient used in quantile-based losses.

Default: 0.5

Lq

i=1Naitiqwii=1Nwi\displaystyle\frac{\sum\limits_{i=1}^N |a_{i} - t_{i}|^q w_i}{\sum\limits_{i=1}^N w_{i}}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

q

The power coefficient.

Valid values are real numbers in the following range:  [1;+)[1; +\infty)

Default: Obligatory parameter

Huber

L(t,a)=i=0Nl(ti,ai)wi,whereL(t, a) = \sum\limits_{i=0}^N l(t_i, a_i) \cdot w_{i} { , where}

l(t,a)={12(ta)2,taδδta12δ2,ta>δl(t,a) = \begin{cases} \frac{1}{2} (t - a)^{2} { , } & |t -a| \leq \delta \\ \delta|t -a| - \frac{1}{2} \delta^{2} { , } & |t -a| > \delta \end{cases}

User-defined parameters:

delta

The δ\delta parameter of the Huber metric.

Default: Obligatory parameter

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

Expectile

i=1Nα1(tiai)(tiai)2wii=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N} |\alpha - 1(t_{i} \leq a_{i})|(t_{i} - a_{i})^2 w_{i} }{\sum\limits_{i=1}^{N} w_{i}}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

alpha

The coefficient used in expectile-based losses.

Default: 0.5

Tweedie

i=1N(eai(2λ)2λtieai(1λ)1λ)wii=1Nwi,where\displaystyle\frac{\sum\limits_{i=1}^{N}\left(\displaystyle\frac{e^{a_{i}(2-\lambda)}}{2-\lambda} - t_{i}\frac{e^{a_{i}(1-\lambda)}}{1-\lambda} \right)\cdot w_{i}}{\sum\limits_{i=1}^{N} w_{i}} { , where}

λ\lambda is the value of the variance_power parameter.

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

variance_power

The variance of the Tweedie distribution.

Supported values are in the range (1;2).

Default: Obligatory parameter

FairLoss

i=1Nc2(tiaicln(tiaic+1))wii=1Nwi,where\displaystyle\frac{\sum\limits_{i=1}^{N} c^2(\frac{|t_{i} - a_{i} |}{c} - \ln(\frac{|t_{i} - a_{i} |}{c} + 1))w_{i}}{\sum\limits_{i=1}^{N} w_{i}} { , where}

cc is the value of the smoothness parameter.

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

use_weights

The smoothness coefficient. Valid values are real values in the following range (0;+)(0; +\infty).

Default: 1.0

NumErrors

The proportion of predictions, for which the difference from the label value exceeds the specified value greater_than.

i=1NI{x}wii=1Nwi,where\displaystyle\frac{\sum\limits_{i=1}^{N} I\{x\} w_{i}}{\sum\limits_{i=1}^{N} w_{i}} { , where}

I{x}={1,aiti>greater_than0,aitigreater_thanI\{x\} = \begin{cases} 1 { , } & |a_{i} - t_{i}| > greater\_than \\ 0 { , } & |a_{i} - t_{i}| \leq greater\_than \end{cases}

User-defined parameters: greater_than

Increase the numerator of the formula if the following inequality is met:

predictionlabel>value|prediction - label|>value

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

SMAPE

100i=1Nwiaiti(ti+ai)/2i=1Nwi\displaystyle\frac{100 \sum\limits_{i=1}^{N}\displaystyle\frac{w_{i} |a_{i} - t_{i} |}{(| t_{i} | + | a_{i} |) / 2}}{\sum\limits_{i=1}^{N} w_{i}}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

R2

1i=1Nwi(aiti)2i=1Nwi(t¯ti)21 - \displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} (a_{i} - t_{i})^{2}}{\sum\limits_{i=1}^{N} w_{i} (\bar{t} - t_{i})^{2}}
t¯\bar{t} is the average label value:
t¯=1Ni=1Nti\bar{t} = \frac{1}{N}\sum\limits_{i=1}^{N}t_{i}

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

MSLE

i=1Nwi(loge(1+ti)loge(1+ai))2i=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} (\log_{e} (1 + t_{i}) - \log_{e} (1 + a_{i}))^{2}}{\sum\limits_{i=1}^{N} w_{i}}

Used for optimization: –

User-defined parameters:

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

MedianAbsoluteError

median(t1a1,...,tiai)median(|t_{1} - a_{1}|, ..., |t_{i} - a_{i}|)

Used for optimization: –

User-defined parameters: –

Used for optimization

Name Optimization
MAE +
MAPE +
Poisson +
Quantile +
RMSE +
RMSEWithUncertainty +
LogLinQuantile +
Lq +
Huber +
Expectile +
Tweedie +
FairLoss -
NumErrors -
SMAPE -
R2 -
MSLE -
MedianAbsoluteError -