Regression: objectives and metrics

Objectives and metrics

MAE

i=1Nwiaitii=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} | a_{i} - t_{i}| }{\sum\limits_{i=1}^{N} w_{i}}

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

MAPE

i=1Nwiaitimax(1,ti)i=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} \displaystyle\frac{|a_{i}- t_{i}|}{\max(1, |t_{i}|)}}{\sum\limits_{i=1}^{N}w_{i}}

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

Poisson

i=1Nwi(eaiaiti)i=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} \left(e^{a_{i}} - a_{i}t_{i}\right)}{\sum\limits_{i=1}^{N}w_{i}}

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

Quantile

i=1N(αI(tiai))(tiai)wii=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N} (\alpha - I(t_{i} \leq a_{i}))(t_{i} - a_{i}) w_{i} }{\sum\limits_{i=1}^{N} w_{i}}

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

alpha

The coefficient used in quantile-based losses.

Default: 0.5

MultiQuantile

i=1Nwiq=1Q(αqI(tiai,q))(tiai,q)i=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} \sum\limits_{q=1}^{Q} (\alpha_{q} - I(t_{i} \leq a_{i,q}))(t_{i} - a_{i,q}) }{\sum\limits_{i=1}^{N} w_{i}}

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

alpha

The vector of coefficients used in multi-quantile loss.

Default: 0.5

RMSE

i=1N(aiti)2wii=1Nwi\displaystyle\sqrt{\displaystyle\frac{\sum\limits_{i=1}^N (a_{i}-t_{i})^2 w_{i}}{\sum\limits_{i=1}^{N}w_{i}}}

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

RMSEWithUncertainty

i=1NwilogN(tiai,0,e2ai,1)i=1Nwi=12log(2π)+i=1Nwi(ai,1+12e2ai,1(tiai,0)2)i=1Nwi\displaystyle-\frac{\sum_{i=1}^N w_i \log N(t_{i} \vert a_{i,0}, e^{2a_{i,1}})}{\sum_{i=1}^{N}w_{i}} = \frac{1}{2}\log(2\pi) +\frac{\sum_{i=1}^N w_i\left(a_{i,1} + \frac{1}{2} e^{-2a_{i,1}}(t_i - a_{i, 0})^2 \right)}{\sum_{i=1}^{N}w_{i}},
where tt is target, a 2-dimensional approx a0a_0 is target predict, a1a_1 is logσ\log \sigma predict, and N(yμ,σ2)=12πσ2exp((yμ)22σ2)N(y\vert \mu,\sigma^2) = \frac{1}{\sqrt{2 \pi\sigma^2}} \exp(-\frac{(y-\mu)^2}{2\sigma^2}) is the probability density function of the normal distribution.

See the Uncertainty section for more details.

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

LogLinQuantile

Depends on the condition for the ratio of the label value and the resulting value:
{i=1Nαtieaiwii=1Nwiti>eaii=1N(1α)tieaiwii=1Nwitieai\begin{cases} \displaystyle\frac{\sum\limits_{i=1}^{N} \alpha |t_{i} - e^{a_{i}} | w_{i}}{\sum\limits_{i=1}^{N} w_{i}} & t_{i} > e^{a_{i}} \\ \displaystyle\frac{\sum\limits_{i=1}^{N} (1 - \alpha) |t_{i} - e^{a_{i}} | w_{i}}{\sum\limits_{i=1}^{N} w_{i}} & t_{i} \leq e^{a_{i}} \end{cases}

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

alpha

The coefficient used in quantile-based losses.

Default: 0.5

Lq

i=1Naitiqwii=1Nwi\displaystyle\frac{\sum\limits_{i=1}^N |a_{i} - t_{i}|^q w_i}{\sum\limits_{i=1}^N w_{i}}

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

q

The power coefficient.

Valid values are real numbers in the following range:  [1;+)[1; +\infty)

Default: Obligatory parameter

Huber

L(t,a)=i=0Nl(ti,ai)wiL(t, a) = \sum\limits_{i=0}^N l(t_i, a_i) \cdot w_{i}

l(t,a)={12(ta)2,taδδta12δ2,ta>δl(t,a) = \begin{cases} \frac{1}{2} (t - a)^{2} { , } & |t -a| \leq \delta \\ \delta|t -a| - \frac{1}{2} \delta^{2} { , } & |t -a| > \delta \end{cases}

User-defined parameters:

delta

The δ\delta parameter of the Huber metric.

Default: Obligatory parameter

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

Expectile

i=1NαI(tiai)(tiai)2wii=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N} |\alpha - I(t_{i} \leq a_{i})|(t_{i} - a_{i})^2 w_{i} }{\sum\limits_{i=1}^{N} w_{i}}

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

alpha

The coefficient used in expectile-based losses.

Default: 0.5

Tweedie

i=1N(eai(2λ)2λtieai(1λ)1λ)wii=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N}\left(\displaystyle\frac{e^{a_{i}(2-\lambda)}}{2-\lambda} - t_{i}\frac{e^{a_{i}(1-\lambda)}}{1-\lambda} \right)\cdot w_{i}}{\sum\limits_{i=1}^{N} w_{i}}

λ\lambda is the value of the variance_power parameter.

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

variance_power

The variance of the Tweedie distribution.

Supported values are in the range (1;2).

Default: Obligatory parameter

LogCosh

i=1Nwilog(cosh(aiti))i=1Nwi\displaystyle\frac{\sum_{i=1}^N w_i \log(\cosh(a_i - t_i))}{\sum_{i=1}^N w_i}

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

FairLoss

i=1Nc2(tiaiclog(tiaic+1))wii=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N} c^2\left(\frac{|t_{i} - a_{i} |}{c} - \log\left(\frac{|t_{i} - a_{i} |}{c} + 1\right)\right)w_{i}}{\sum\limits_{i=1}^{N} w_{i}}

cc is the value of the smoothness parameter.

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

use_weights

The smoothness coefficient. Valid values are real values in the following range (0;+)(0; +\infty).

Default: 1.0

NumErrors

The proportion of predictions, for which the difference from the label value exceeds the specified value greater_than.

i=1NI(aitigreater_than)wii=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N} I(|a_{i} - t_{i}|\geq \text{greater\_than}) w_{i}}{\sum\limits_{i=1}^{N} w_{i}}

User-defined parameters: greater_than

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

SMAPE

100i=1Nwiaiti(ti+ai)/2i=1Nwi\displaystyle\frac{100 \sum\limits_{i=1}^{N}\displaystyle\frac{w_{i} |a_{i} - t_{i} |}{(| t_{i} | + | a_{i} |) / 2}}{\sum\limits_{i=1}^{N} w_{i}}

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

R2

1i=1Nwi(aiti)2i=1Nwi(tˉti)21 - \displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} (a_{i} - t_{i})^{2}}{\sum\limits_{i=1}^{N} w_{i} (\bar{t} - t_{i})^{2}}
tˉ\bar{t} is the average label value:
tˉ=1Ni=1Nti\bar{t} = \frac{1}{N}\sum\limits_{i=1}^{N}t_{i}

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

MSLE

i=1Nwi(log(1+ti)log(1+ai))2i=1Nwi\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} (\log (1 + t_{i}) - \log (1 + a_{i}))^{2}}{\sum\limits_{i=1}^{N} w_{i}}

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

MedianAbsoluteError

median(t1a1,...,tNaN)\displaystyle\text{median}(|t_{1} - a_{1}|, ..., |t_{N} - a_{N}|)

Can't be used for optimization. See more.

User-defined parameters

No.

Cox

ti>0(ailogtjtiexp(aj))\displaystyle\sum\limits_{t_i > 0}\left( a_i - \log\sum\limits_{|t_j| \ge t_i} \exp(a_j)\right)

Labels ti>0t_i > 0 mean occurence of the event at time tit_i, and labels ti<0t_i < 0 mean absence of the event at time ti|t_i|.

Predictions aia_i are hazard rates.

Usage information See more.

User-defined parameters

No.

SurvivalAft

ti,0=ti,1log(f(ϵ(ti,0,ai))+ti,0ti,1log(F(ϵ(ti,1,ai))F(ϵ(ti,0,ai)))\displaystyle\sum\limits_{t_{i,0} = t_{i,1}} \log\left(f(\epsilon(t_{i,0}, a_i)\right) + \sum\limits_{t_{i,0} \ne t_{i,1}} \log \left(F(\epsilon(t_{i,1}, a_i)) - F(\epsilon(t_{i,0}, a_i))\right)

Observation interval is [ti,0,ti,1][t_{i,0}, t_{i,1}] for ti,11t_{i,1} \ne -1, and [ti,0,)[t_{i,0}, \infty) for ti,1=1t_{i,1} = -1.

Predictions aia_i are hazard rates.

Helper ϵ(t,a)=(logta)/σ\epsilon(t, a) = (\log t - a)/\sigma for t1t \ne -1, and ϵ(1,a)=\epsilon(-1, a) = \infty, is hazard prediction error.

Coefficient σ\sigma is scale of hazard prediction error, specified by scale parameter.

Functions ff and FF are probability density and cumulative distribution, specified by dist parameter.

dist

Guessed distribution of hazard prediction error.

Possible values: Normal, Extreme, Logistic.

dist FF ff
Normal 12(1+erf(z2))\displaystyle\frac{1}{2}\left(1+\text{erf}\left( \frac{z}{\sqrt{2}}\right)\right) ez2/22π\displaystyle\frac{e^{-z^2/2}}{\sqrt{2\pi}}
Logistic ez1+ez\displaystyle\frac{e^z}{1+e^z} ez(1+ez)2\displaystyle\frac{e^z}{(1+e^z)^2}
Extreme 1eez\displaystyle 1-e^{-e^z} ezeez\displaystyle e^ze^{-e^z}

Default: Normal

scale

Scale of hazard prediction error.

Default: 1.0

Usage information See more.

User-defined parameters

No.

Used for optimization

Name Optimization GPU Support
MAE + +
MAPE + +
Poisson + +
Quantile + +
MultiQuantile + -
RMSE + +
RMSEWithUncertainty + -
LogLinQuantile + +
Lq + +
Huber + +
Expectile + +
Tweedie + +
LogCosh + -
Cox + -
SurvivalAft + -
FairLoss - -
NumErrors - +
SMAPE - -
R2 - -
MSLE - -
MedianAbsoluteError - -