Objectives and metrics

This section contains basic information regarding the supported metrics for various machine learning problems.

Refer to the Variables used in formulas section for the description of commonly used variables in the listed metrics.

Metrics can be calculated during the training or separately from the training for a specified model. The calculated values are written to files and can be plotted by visualization tools (both during and after the training) for further analysis.

User-defined parameters

Some metrics provide user-defined parameters. These parameters must be set together with the metric name when it is being specified.

The parameters for each metric are set in the following format:

<Metric>[:<parameter 1>=<value>;..;<parameter N>=<value>]

The supported parameters vary from one metric to another and are listed alongside the corresponding descriptions.

Usage examples

Quantile:alpha=0.1

List of most important parameters

The following table contains the description of parameters that are used in several metrics. The default values vary from one metric to another and are listed alongside the corresponding descriptions.

Parameter: use_weights

Description

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Note

This parameter cannot be used with the optimized objective. If weights are present, they are necessarily used to calculate the optimized objective. This behaviour cannot be disabled.

Parameter: top

Description

The number of top samples in a group that are used to calculate the ranking metric. Top samples are either the samples with the largest approx values or the ones with the lowest target values if approx values are the same.

Enable, disable and configure metrics calculation

The calculation of metrics can be resource-intensive. It creates a bottleneck in some cases, for example, if many metrics are calculated during the training or the computation is performed on GPU.

The training can be sped up by disabling the calculation of some metrics for the training dataset. Use the hints=skip_train~true parameter to disable the calculation of the specified metrics.

Note

The calculation of some metrics is disabled by default for the training dataset to speed up the training. Use the hints=skip_train~false parameter to enable the calculation.

Metrics that are not calculated by default for the train dataset
  • PFound
  • YetiRank
  • NDCG
  • YetiRankPairwise
  • AUC
  • NormalizedGini
  • FilteredDCG
  • DCG
Usage examples

Enable the calculation of the AUC metric:

AUC:hints=skip_train~false

Disable the calculation of the Logloss metric:

Logloss:hints=skip_train~true

Another way to speed up the training is to set up the frequency of iterations to calculate the values of metrics. Use one of the following parameters:

Command-line version parameters: --metric-period

Python parameters: metric_period

R parameters: metric_period

For example, use the following parameter in Python or R to calculate metrics once per 50 iterations:

metric_period=50