Classification: objectives and metrics
Objectives and metrics
Logloss
Usage information See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
CrossEntropy
Usage information See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
Precision
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
Recall
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
F
Can't be used for optimization. See more.
User-defined parameters
beta
The parameter of the F metric.
Valid values are real numbers in the following range: .
Default: This parameter is obligatory (the default value is not defined)
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
F1
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
BalancedAccuracy
User-defined parameters:
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
BalancedErrorRate
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
MCC
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
Accuracy
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
CtrFactor
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
AUC
The calculation of this metric is disabled by default for the training dataset to speed up the training. Use the hints=skip_train~false parameter to enable the calculation.
Classic
The sum is calculated on all pairs of objects such that:
Refer to the Wikipedia article for details.
If the target type is not binary, then every object with target value and weight is replaced with two objects for the metric calculation:
- with weight and target value 1
- with weight and target value 0.
Target values must be in the range [0; 1].
Ranking
The sum is calculated on all pairs of objects such that:
User-defined parameters
type
The type of AUC. Defines the metric calculation principles.
Default: Ranking.
Possible values: Classic, Ranking.
Examples: AUC:type=Classic, AUC:type=Ranking.
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: False.
Examples: QueryAUC:type=Ranking;use_weights=False.
QueryAUC
Classic type
The sum is calculated on all pairs of objects such that:
Refer to the Wikipedia article for details.
If the target type is not binary, then every object with target value and weight is replaced with two objects for the metric calculation:
- with weight and target value 1
- with weight and target value 0.
Target values must be in the range [0; 1].
Ranking type
The sum is calculated on all pairs of objects such that:
Can't be used for optimization. See more.
User-defined parameters
type
The type of QueryAUC. Defines the metric calculation principles.
Default: Ranking.
Possible values: Classic, Ranking.
Examples: QueryAUC:type=Classic, QueryAUC:type=Ranking.
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: False.
Examples: QueryAUC:type=Ranking;use_weights=False.
PRAUC
PRAUC is the area under the curve vs for where and are defined as follows.
Above , , are weights of the true positive, false positive, and false negative samples, respectively.
To calculate PRAUC for a binary classification model, specify type Classic.
In this case, , etc.
To calculate PRAUC for a multi-classification model, specify type OneVsAll.
In this case, positive samples are samples having class 0, all other samples are negative, and , etc.
type
The type of PRAUC. Defines the metric calculation principles.
Type Classic is compatible with binary classification models.
Type OneVsAll is compatible with multi-classification models.
Default: Classic.
Possible values: Classic, OneVsAll.
Examples: PRAUC:type=Classic, PRAUC:type=OneVsAll.
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: False.
Examples: PRAUC:type=Classic;use_weights=False.
NormalizedGini
See AUC.
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
BrierScore
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
HingeLoss
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
HammingLoss
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
ZeroOneLoss
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
Kappa
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
WKappa
See the formula on page 3 of the A note on the linearly weighted kappa coefficient for ordinal scales paper.
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
LogLikelihoodOfPrediction
The calculation consists of the following steps:
-
Define the sum of weights () and the mean target ():
-
Denote log-likelihood of a constant prediction:
-
Calculate LogLikelihoodOfPrediction (), which reflects how the likelihood () differs from the constant prediction:
Can't be used for optimization. See more.
User-defined parameters
use_weights
Use object/group weights to calculate metrics if the specified value is true
and set all weights to 1
regardless of the input data if the specified value is false
.
Default: true
Used for optimization
| Name | Optimization | GPU Support |
|---|---|---|
| Logloss | + | + |
| CrossEntropy | + | + |
| Precision | - | + |
| Recall | - | + |
| F | - | - |
| F1 | - | + |
| BalancedAccuracy | - | - |
| BalancedErrorRate | - | - |
| MCC | - | + |
| Accuracy | - | + |
| CtrFactor | - | - |
| AUC | - | - |
| QueryAUC | - | - |
| NormalizedGini | - | - |
| BrierScore | - | - |
| HingeLoss | - | - |
| HammingLoss | - | - |
| ZeroOneLoss | - | + |
| Kappa | - | - |
| WKappa | - | - |
| LogLikelihoodOfPrediction | - | - |