Objectives and metrics
This section contains basic information regarding the supported metrics for various machine learning problems.
Refer to the Variables used in formulas section for the description of commonly used variables in the listed metrics.
Metrics can be calculated during the training or separately from the training for a specified model. The calculated values are written to files and can be plotted by visualization tools (both during and after the training) for further analysis.
User-defined parameters
Some metrics provide user-defined parameters. These parameters must be set together with the metric name when it is being specified.
The parameters for each metric are set in the following format:
<Metric>[:<parameter 1>=<value>;..;<parameter N>=<value>]
The supported parameters vary from one metric to another and are listed alongside the corresponding descriptions.
Usage examples
Quantile:alpha=0.1
List of most important parameters
The following table contains the description of parameters that are used in several metrics. The default values vary from one metric to another and are listed alongside the corresponding descriptions.
Parameter: use_weights
Description
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Note
This parameter cannot be used with the optimized objective. If weights are present, they are necessarily used to calculate the optimized objective. This behaviour cannot be disabled.
Parameter: top
Description
The number of top samples in a group that are used to calculate the ranking metric. Top samples are either the samples with the largest approx values or the ones with the lowest target values if approx values are the same.
Enable, disable and configure metrics calculation
The calculation of metrics can be resource-intensive. It creates a bottleneck in some cases, for example, if many metrics are calculated during the training or the computation is performed on GPU.
The training can be sped up by disabling the calculation of some metrics for the training dataset. Use the hints=skip_train~true parameter to disable the calculation of the specified metrics.
Note
The calculation of some metrics is disabled by default for the training dataset to speed up the training. Use the hints=skip_train~false parameter to enable the calculation.
Metrics that are not calculated by default for the train dataset
- PFound
- YetiRank
- NDCG
- YetiRankPairwise
- AUC
- NormalizedGini
- FilteredDCG
- DCG
Usage examples
Enable the calculation of the AUC metric:
AUC:hints=skip_train~false
Disable the calculation of the Logloss metric:
Logloss:hints=skip_train~true
Another way to speed up the training is to set up the frequency of iterations to calculate the values of metrics. Use one of the following parameters:
Command-line version parameters: --metric-period
Python parameters: metric_period
R parameters: metric_period
For example, use the following parameter in Python or R to calculate metrics once per 50 iterations:
metric_period=50