Regression: objectives and metrics

• Objectives and metrics
• Used for optimization

Objectives and metricsObjectives and metrics

MAEMAE

$\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} | a_{i} - t_{i}| }{\sum\limits_{i=1}^{N} w_{i}}$

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

MAPEMAPE

$\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} \displaystyle\frac{|a_{i}- t_{i}|}{\max(1, |t_{i}|)}}{\sum\limits_{i=1}^{N}w_{i}}$

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

PoissonPoisson

$\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} \left(e^{a_{i}} - a_{i}t_{i}\right)}{\sum\limits_{i=1}^{N}w_{i}}$

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

QuantileQuantile

$\displaystyle\frac{\sum\limits_{i=1}^{N} (\alpha - I(t_{i} \leq a_{i}))(t_{i} - a_{i}) w_{i} }{\sum\limits_{i=1}^{N} w_{i}}$

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

alpha

The coefficient used in quantile-based losses.

Default: 0.5

MultiQuantileMultiQuantile

$\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} \sum\limits_{q=1}^{Q} (\alpha_{q} - I(t_{i} \leq a_{i,q}))(t_{i} - a_{i,q}) }{\sum\limits_{i=1}^{N} w_{i}}$

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

alpha

The vector of coefficients used in multi-quantile loss.

Default: 0.5

RMSERMSE

$\displaystyle\sqrt{\displaystyle\frac{\sum\limits_{i=1}^N (a_{i}-t_{i})^2 w_{i}}{\sum\limits_{i=1}^{N}w_{i}}}$

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

RMSEWithUncertaintyRMSEWithUncertainty

$\displaystyle-\frac{\sum_{i=1}^N w_i \log N(t_{i} \vert a_{i,0}, e^{2a_{i,1}})}{\sum_{i=1}^{N}w_{i}} = \frac{1}{2}\log(2\pi) +\frac{\sum_{i=1}^N w_i\left(a_{i,1} + \frac{1}{2} e^{-2a_{i,1}}(t_i - a_{i, 0})^2 \right)}{\sum_{i=1}^{N}w_{i}}$,
where $t$ is target, a 2-dimensional approx $a_0$ is target predict, $a_1$ is $\log \sigma$ predict, and $N(y\vert \mu,\sigma^2) = \frac{1}{\sqrt{2 \pi\sigma^2}} \exp(-\frac{(y-\mu)^2}{2\sigma^2})$ is the probability density function of the normal distribution.

See the Uncertainty section for more details.

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

LogLinQuantileLogLinQuantile

Depends on the condition for the ratio of the label value and the resulting value:
$\begin{cases} \displaystyle\frac{\sum\limits_{i=1}^{N} \alpha |t_{i} - e^{a_{i}} | w_{i}}{\sum\limits_{i=1}^{N} w_{i}} & t_{i} > e^{a_{i}} \\ \displaystyle\frac{\sum\limits_{i=1}^{N} (1 - \alpha) |t_{i} - e^{a_{i}} | w_{i}}{\sum\limits_{i=1}^{N} w_{i}} & t_{i} \leq e^{a_{i}} \end{cases}$

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

alpha

The coefficient used in quantile-based losses.

Default: 0.5

LqLq

$\displaystyle\frac{\sum\limits_{i=1}^N |a_{i} - t_{i}|^q w_i}{\sum\limits_{i=1}^N w_{i}}$

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

q

The power coefficient.

Valid values are real numbers in the following range:  $[1; +\infty)$

Default: Obligatory parameter

HuberHuber

$L(t, a) = \sum\limits_{i=0}^N l(t_i, a_i) \cdot w_{i}$

$l(t,a) = \begin{cases} \frac{1}{2} (t - a)^{2} { , } & |t -a| \leq \delta \\ \delta|t -a| - \frac{1}{2} \delta^{2} { , } & |t -a| > \delta \end{cases}$

User-defined parameters:

delta

The $\delta$ parameter of the Huber metric.

Default: Obligatory parameter

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

ExpectileExpectile

$\displaystyle\frac{\sum\limits_{i=1}^{N} |\alpha - I(t_{i} \leq a_{i})|(t_{i} - a_{i})^2 w_{i} }{\sum\limits_{i=1}^{N} w_{i}}$

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

alpha

The coefficient used in expectile-based losses.

Default: 0.5

TweedieTweedie

$\displaystyle\frac{\sum\limits_{i=1}^{N}\left(\displaystyle\frac{e^{a_{i}(2-\lambda)}}{2-\lambda} - t_{i}\frac{e^{a_{i}(1-\lambda)}}{1-\lambda} \right)\cdot w_{i}}{\sum\limits_{i=1}^{N} w_{i}}$

$\lambda$ is the value of the variance_power parameter.

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

variance_power

The variance of the Tweedie distribution.

Supported values are in the range (1;2).

Default: Obligatory parameter

LogCoshLogCosh

$\displaystyle\frac{\sum_{i=1}^N w_i \log(\cosh(a_i - t_i))}{\sum_{i=1}^N w_i}$

Usage information See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

FairLossFairLoss

$\displaystyle\frac{\sum\limits_{i=1}^{N} c^2\left(\frac{|t_{i} - a_{i} |}{c} - \log\left(\frac{|t_{i} - a_{i} |}{c} + 1\right)\right)w_{i}}{\sum\limits_{i=1}^{N} w_{i}}$

$c$ is the value of the smoothness parameter.

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

use_weights

The smoothness coefficient. Valid values are real values in the following range $(0; +\infty)$.

Default: 1.0

NumErrorsNumErrors

The proportion of predictions, for which the difference from the label value exceeds the specified value greater_than.

$\displaystyle\frac{\sum\limits_{i=1}^{N} I(|a_{i} - t_{i}|\geq \text{greater\_than}) w_{i}}{\sum\limits_{i=1}^{N} w_{i}}$

User-defined parameters: greater_than

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

SMAPESMAPE

$\displaystyle\frac{100 \sum\limits_{i=1}^{N}\displaystyle\frac{w_{i} |a_{i} - t_{i} |}{(| t_{i} | + | a_{i} |) / 2}}{\sum\limits_{i=1}^{N} w_{i}}$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

R2R2

$1 - \displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} (a_{i} - t_{i})^{2}}{\sum\limits_{i=1}^{N} w_{i} (\bar{t} - t_{i})^{2}}$
$\bar{t}$ is the average label value:
$\bar{t} = \frac{1}{N}\sum\limits_{i=1}^{N}t_{i}$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

MSLEMSLE

$\displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} (\log (1 + t_{i}) - \log (1 + a_{i}))^{2}}{\sum\limits_{i=1}^{N} w_{i}}$

Can't be used for optimization. See more.

User-defined parameters

use_weights

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

MedianAbsoluteErrorMedianAbsoluteError

$\displaystyle\text{median}(|t_{1} - a_{1}|, ..., |t_{N} - a_{N}|)$

Can't be used for optimization. See more.

User-defined parameters

No.

CoxCox

$\displaystyle\sum\limits_{t_i > 0}\left( a_i - \log\sum\limits_{|t_j| \ge t_i} \exp(a_j)\right)$

Labels $t_i > 0$ mean occurence of the event at time $t_i$, and labels $t_i < 0$ mean absence of the event at time $|t_i|$.

Predictions $a_i$ are hazard rates.

Usage information See more.

User-defined parameters

No.

SurvivalAftSurvivalAft

$\displaystyle\sum\limits_{t_{i,0} = t_{i,1}} \log\left(f(\epsilon(t_{i,0}, a_i)\right) + \sum\limits_{t_{i,0} \ne t_{i,1}} \log \left(F(\epsilon(t_{i,1}, a_i)) - F(\epsilon(t_{i,0}, a_i))\right)$

Observation interval is $[t_{i,0}, t_{i,1}]$ for $t_{i,1} \ne -1$, and $[t_{i,0}, \infty)$ for $t_{i,1} = -1$.

Predictions $a_i$ are hazard rates.

Helper $\epsilon(t, a) = (\log t - a)/\sigma$ for $t \ne -1$, and $\epsilon(-1, a) = \infty$, is hazard prediction error.

Coefficient $\sigma$ is scale of hazard prediction error, specified by scale parameter.

Functions $f$ and $F$ are probability density and cumulative distribution, specified by dist parameter.

dist

Guessed distribution of hazard prediction error.

Possible values: Normal, Extreme, Logistic.

dist	$F$	$f$
Normal	$\displaystyle\frac{1}{2}\left(1+\text{erf}\left( \frac{z}{\sqrt{2}}\right)\right)$	$\displaystyle\frac{e^{-z^2/2}}{\sqrt{2\pi}}$
Logistic	$\displaystyle\frac{e^z}{1+e^z}$	$\displaystyle\frac{e^z}{(1+e^z)^2}$
Extreme	$\displaystyle 1-e^{-e^z}$	$\displaystyle e^ze^{-e^z}$

Default: Normal

scale

Scale of hazard prediction error.

Default: 1.0

Usage information See more.

User-defined parameters

No.

Used for optimizationUsed for optimization

Name	Optimization	GPU Support
MAE	+	+
MAPE	+	+
Poisson	+	+
Quantile	+	+
MultiQuantile	+	-
RMSE	+	+
RMSEWithUncertainty	+	+
LogLinQuantile	+	+
Lq	+	+
Huber	+	+
Expectile	+	+
Tweedie	+	+
LogCosh	+	-
Cox	+	-
SurvivalAft	+	-
FairLoss	-	-
NumErrors	-	+
SMAPE	-	-
R2	-	-
MSLE	-	-
MedianAbsoluteError	-	-