eval_metric

Calculate the specified metric on raw approximated values of the formula and label values.

Method call format

eval_metric(label,
            approx,
            metric,
            weight=None,
            group_id=None,
            subgroup_id=None, 
            pairs=None, 
            thread_count=-1)

Parameters

Parameter Possible types Description Default value
label
  • list
  • numpy.array
  • pandas.DataFrame
  • pandas.Series

A list of target variables (in other words, the label values of the objects).

Required parameter
approx
  • list
  • numpy.array
  • pandas.DataFrame
  • pandas.Series

A list of approximate values for all input objects.

Required parameter
metric string

The evaluation metric to calculate.

Supported metrics
  • RMSE
  • Logloss
  • MAE
  • CrossEntropy
  • Quantile
  • LogLinQuantile
  • Lq
  • MultiClass
  • MultiClassOneVsAll
  • MAPE
  • Poisson
  • PairLogit
  • PairLogitPairwise
  • QueryRMSE
  • QuerySoftMax
  • SMAPE
  • Recall
  • Precision
  • F1
  • TotalF1
  • Accuracy
  • BalancedAccuracy
  • BalancedErrorRate
  • Kappa
  • WKappa
  • LogLikelihoodOfPrediction
  • AUC
  • R2
  • FairLoss
  • NumErrors
  • MCC
  • BrierScore
  • HingeLoss
  • HammingLoss
  • ZeroOneLoss
  • MSLE
  • MedianAbsoluteError
  • Huber
  • Expectile
  • PairAccuracy
  • AverageGain
  • PFound
  • NDCG
  • DCG
  • FilteredDCG
  • NormalizedGini
  • PrecisionAt
  • RecallAt
  • MAP
  • CtrFactor
  • YetiRank
  • YetiRankPairwise
Required parameter
weight
  • list
  • numpy.array
  • pandas.DataFrame
  • pandas.Series
The weights of objects. None
group_id
  • list
  • numpy.array
  • pandas.DataFrame
  • pandas.Series
Group identifiers for all input objects. Supported identifier types are:
  • int
  • string types (string or unicode for Python 2 and bytes or string for Python 3).
None
subgroup_id
  • list
  • numpy.array
Subgroup identifiers for all input objects. None
pairs
  • list
  • numpy.array
  • pandas.DataFrame

The pairs description in the form of a two-dimensional matrix of shape N by 2:

  • N is the number of pairs.
  • The first element of the pair is the zero-based index of the winner object from the input dataset for pairwise comparison.
  • The second element of the pair is the zero-based index of the loser object from the input dataset for pairwise comparison.

This information is used for calculation and optimization of Pairwise metrics .

None
string

The path to the input file that contains the pairs description.

This information is used for calculation and optimization of Pairwise metrics .

None
thread_count int

The number of threads to use.

Optimizes the speed of execution. This parameter doesn't affect results.

-1 (the number of threads is equal to the number of processor cores)
Parameter Possible types Description Default value
label
  • list
  • numpy.array
  • pandas.DataFrame
  • pandas.Series

A list of target variables (in other words, the label values of the objects).

Required parameter
approx
  • list
  • numpy.array
  • pandas.DataFrame
  • pandas.Series

A list of approximate values for all input objects.

Required parameter
metric string

The evaluation metric to calculate.

Supported metrics
  • RMSE
  • Logloss
  • MAE
  • CrossEntropy
  • Quantile
  • LogLinQuantile
  • Lq
  • MultiClass
  • MultiClassOneVsAll
  • MAPE
  • Poisson
  • PairLogit
  • PairLogitPairwise
  • QueryRMSE
  • QuerySoftMax
  • SMAPE
  • Recall
  • Precision
  • F1
  • TotalF1
  • Accuracy
  • BalancedAccuracy
  • BalancedErrorRate
  • Kappa
  • WKappa
  • LogLikelihoodOfPrediction
  • AUC
  • R2
  • FairLoss
  • NumErrors
  • MCC
  • BrierScore
  • HingeLoss
  • HammingLoss
  • ZeroOneLoss
  • MSLE
  • MedianAbsoluteError
  • Huber
  • Expectile
  • PairAccuracy
  • AverageGain
  • PFound
  • NDCG
  • DCG
  • FilteredDCG
  • NormalizedGini
  • PrecisionAt
  • RecallAt
  • MAP
  • CtrFactor
  • YetiRank
  • YetiRankPairwise
Required parameter
weight
  • list
  • numpy.array
  • pandas.DataFrame
  • pandas.Series
The weights of objects. None
group_id
  • list
  • numpy.array
  • pandas.DataFrame
  • pandas.Series
Group identifiers for all input objects. Supported identifier types are:
  • int
  • string types (string or unicode for Python 2 and bytes or string for Python 3).
None
subgroup_id
  • list
  • numpy.array
Subgroup identifiers for all input objects. None
pairs
  • list
  • numpy.array
  • pandas.DataFrame

The pairs description in the form of a two-dimensional matrix of shape N by 2:

  • N is the number of pairs.
  • The first element of the pair is the zero-based index of the winner object from the input dataset for pairwise comparison.
  • The second element of the pair is the zero-based index of the loser object from the input dataset for pairwise comparison.

This information is used for calculation and optimization of Pairwise metrics .

None
string

The path to the input file that contains the pairs description.