Calculate metrics

Purpose

Calculate metrics for a given dataset using a previously trained model.

Execution format

catboost eval-metrics --metrics <comma-separated list of metrics> [optional parameters]

Options


Option	Description	Default value
-m --model-path	The name of the input file with the description of the model obtained as the result of training.	model.bin
--model-format	The format of the input model. Possible values: CatboostBinary. AppleCoreML (only datasets without categorical features are currently supported). json (multiclassification models are not currently supported). Refer to the CatBoost JSON model tutorial for format details.	CatboostBinary
--input-path	The name of the input file with the dataset description.	input.tsv
--column-description --cd	The path to the input file that contains the column descriptions.	If omitted, it is assumed that the first column in the file with the dataset description defines the label value, and the other columns are the values of numerical features.
-o --output-path	The name of the output file that contains the resulting values of the model for the input objects. The format depends on the problem being solved.	output.tsv
-T --thread-count	The number of threads to use during training. Optimizes the speed of execution. This parameter doesn't affect results.	The number of processor cores
--delimiter	The delimiter character used to separate the data in the dataset description input file. Only single char delimiters are supported. If the specified value contains more than one character, only the first one is used.	The input data is assumed to be tab-separated
--has-header	False	False
--ntree-start	To reduce the number of trees to use when the model is applied or the metrics are calculated, set the range of the tree indices to `[--ntree-start; --ntree-end)` and the step of the trees to use to --eval-period. This parameter defines the index of the first tree to be used when applying the model or calculating the metrics (the inclusive left border of the range). Indices are zero-based.	0
--ntree-end	To reduce the number of trees to use when the model is applied or the metrics are calculated, set the range of the tree indices to `[--ntree-start; --ntree-end)` and the step of the trees to use to --eval-period. This parameter defines the index of the first tree not to be used when applying the model or calculating the metrics (the exclusive right border of the range). Indices are zero-based.	0 (the index of the last tree to use equals to the number of trees in the model minus one)
--eval-period	To reduce the number of trees to use when the model is applied or the metrics are calculated, set the range of the tree indices to `[--ntree-start; --ntree-end)` and the step of the trees to use to --eval-period. This parameter defines the step of the trees to use for calculating the metrics. In this mode the results for the `(n*i)`-th tree of the model are calculated taking into consideration only the trees in the range [--ntree-start; --ntree-end). The specified value `n` defines the size of the range of trees to use. The approximate values for the last period are calculated using all trees in the provided segment.	0 (the index of the last tree to use equals to the number of trees in the model minus one)
--metrics	A comma-separated list of metrics to be calculated. Possible values: RMSE Logloss MAE CrossEntropy Quantile LogLinQuantile Lq MultiClass MultiClassOneVsAll MAPE Poisson PairLogit PairLogitPairwise QueryRMSE QuerySoftMax SMAPE Recall Precision F1 TotalF1 Accuracy BalancedAccuracy BalancedErrorRate Kappa WKappa LogLikelihoodOfPrediction AUC R2 NumErrors MCC BrierScore HingeLoss HammingLoss ZeroOneLoss MSLE MedianAbsoluteError PairAccuracy AverageGain PFound NDCG PrecisionAt RecallAt MAP CtrFactor For example, if the AUC and Logloss metrics should be calculated, use the following construction: `--metrics AUC,Logloss`	Required parameter
--result-dir	The directory for storing the files generated during metric calculation.	None (current directory)
--block-size	The computing block size.	32768
--tmp-dir	The directory for storing temporary files that are generated if non-additive metrics are calculated. By default, the directory is generated inside the current one at the start of calculation, and it is removed when the calculation is complete. Otherwise the specified value is used.	- (the directory is generated)
--verbose	Verbose output to stdout.	False


Option	Description	Default value
-m --model-path	The name of the input file with the description of the model obtained as the result of training.	model.bin
--model-format	The format of the input model. Possible values: CatboostBinary. AppleCoreML (only datasets without categorical features are currently supported). json (multiclassification models are not currently supported). Refer to the CatBoost JSON model tutorial for format details.	CatboostBinary
--input-path	The name of the input file with the dataset description.	input.tsv
--column-description --cd	The path to the input file that contains the column descriptions.	If omitted, it is assumed that the first column in the file with the dataset description defines the label value, and the other columns are the values of numerical features.
-o --output-path	The name of the output file that contains the resulting values of the model for the input objects. The format depends on the problem being solved.	output.tsv
-T --thread-count	The number of threads to use during training. Optimizes the speed of execution. This parameter doesn't affect results.	The number of processor cores
--delimiter	The delimiter character used to separate the data in the dataset description input file. Only single char delimiters are supported. If the specified value contains more than one character, only the first one is used.	The input data is assumed to be tab-separated
--has-header	False	False
--ntree-start	To reduce the number of trees to use when the model is applied or the metrics are calculated, set the range of the tree indices to `[--ntree-start; --ntree-end)` and the step of the trees to use to --eval-period. This parameter defines the index of the first tree to be used when applying the model or calculating the metrics (the inclusive left border of the range). Indices are zero-based.	0
--ntree-end	To reduce the number of trees to use when the model is applied or the metrics are calculated, set the range of the tree indices to `[--ntree-start; --ntree-end)` and the step of the trees to use to --eval-period. This parameter defines the index of the first tree not to be used when applying the model or calculating the metrics (the exclusive right border of the range). Indices are zero-based.	0 (the index of the last tree to use equals to the number of trees in the model minus one)
--eval-period	To reduce the number of trees to use when the model is applied or the metrics are calculated, set the range of the tree indices to `[--ntree-start; --ntree-end)` and the step of the trees to use to --eval-period. This parameter defines the step of the trees to use for calculating the metrics. In this mode the results for the `(n*i)`-th tree of the model are calculated taking into consideration only the trees in the range [--ntree-start; --ntree-end). The specified value `n` defines the size of the range of trees to use. The approximate values for the last period are calculated using all trees in the provided segment.	0 (the index of the last tree to use equals to the number of trees in the model minus one)
--metrics	A comma-separated list of metrics to be calculated. Possible values: RMSE Logloss MAE CrossEntropy Quantile LogLinQuantile Lq MultiClass MultiClassOneVsAll MAPE Poisson PairLogit PairLogitPairwise QueryRMSE QuerySoftMax SMAPE Recall Precision F1 TotalF1 Accuracy BalancedAccuracy BalancedErrorRate Kappa WKappa LogLikelihoodOfPrediction AUC R2 NumErrors MCC BrierScore HingeLoss HammingLoss ZeroOneLoss MSLE MedianAbsoluteError PairAccuracy AverageGain PFound NDCG PrecisionAt RecallAt MAP CtrFactor For example, if the AUC and Logloss metrics should be calculated, use the following construction: `--metrics AUC,Logloss`	Required parameter
--result-dir	The directory for storing the files generated during metric calculation.	None (current directory)
--block-size	The computing block size.	32768
--tmp-dir	The directory for storing temporary files that are generated if non-additive metrics are calculated. By default, the directory is generated inside the current one at the start of calculation, and it is removed when the calculation is complete. Otherwise the specified value is used.	- (the directory is generated)
--verbose	Verbose output to stdout.	False