Ranking

Pairwise metrics

If pairs are not present in PairLogit or PairLogitPairwise modes, then they are generated automatically in each group using object label values.

If pairs are present, then object label values are not taken into account when defining the winner object in each pair. Regardless of the label values, the first object in the pair is considered a winner.


Name	Used for optimization	User-defined parameters	Formula and/or description
PairLogit	+	use_weights Default: true max_pairs Default: All possible pairs are generated in each group	Calculation principles $\text{[math]}$ Note. The object weights are not used to calculate and optimize the value of this metric. The weights of object pairs are used instead. Use the corresponding column of the Pair description file to change the importance of a certain pair.
PairLogitPairwise	+	use_weights Default: true max_pairs Default: All possible pairs are generated in each group	Calculation principles $\text{[math]}$ This metric may give more accurate results on large datasets compared to PairLogit but it is calculated significantly slower. This technique is described in the Winning The Transfer Learning Track of Yahoo!’s Learning To Rank Challenge with YetiRank paper. Note. The object weights are not used to calculate and optimize the value of this metric. The weights of object pairs are used instead. Use the corresponding column of the Pair description file to change the importance of a certain pair.
PairAccuracy	–	use_weights Default: true	Calculation principles $\text{[math]}$ Note. The object weights are not used to calculate the value of this metric. The weights of object pairs are used instead. Use the corresponding column of the Pair description file to change the importance of a certain pair.


Name	Used for optimization	User-defined parameters	Formula and/or description
PairLogit	+	use_weights Default: true max_pairs Default: All possible pairs are generated in each group	Calculation principles $\text{[math]}$ Note. The object weights are not used to calculate and optimize the value of this metric. The weights of object pairs are used instead. Use the corresponding column of the Pair description file to change the importance of a certain pair.
PairLogitPairwise	+	use_weights Default: true max_pairs Default: All possible pairs are generated in each group	Calculation principles $\text{[math]}$ This metric may give more accurate results on large datasets compared to PairLogit but it is calculated significantly slower. This technique is described in the Winning The Transfer Learning Track of Yahoo!’s Learning To Rank Challenge with YetiRank paper. Note. The object weights are not used to calculate and optimize the value of this metric. The weights of object pairs are used instead. Use the corresponding column of the Pair description file to change the importance of a certain pair.
PairAccuracy	–	use_weights Default: true	Calculation principles $\text{[math]}$ Note. The object weights are not used to calculate the value of this metric. The weights of object pairs are used instead. Use the corresponding column of the Pair description file to change the importance of a certain pair.

Groupwise metrics


Name	Used for optimization	User-defined parameters	Formula and/or description
YetiRank *	+	decay Default: 0.99 permutations Default: 10 use_weights Default: true	An approximation of ranking metrics (such as NDCG and PFound). Allows to use ranking metrics for optimization. The value of this metric can not be calculated. By default, the value of the PFound metric is written to output data if YetiRank is optimized. This metric gives less accurate results on big datasets compared to YetiRankPairwise but it is significantly faster. Note. The object weights are not used to optimize this metric. The group weights are used instead. This objective is used to optimize PairLogit. Automatically generated object pairs are used for this purpose. These pairs are generated independently for each object group. Use the Group weights file or the GroupWeight column of the Column descriptions file to change the group importance. In this case, the weight of each generated pair is multiplied by the value of the corresponding group weight.
YetiRankPairwise *	+	decay Default: 0.99 permutations Default: 10 use_weights Default: true	An approximation of ranking metrics (such as NDCG and PFound). Allows to use ranking metrics for optimization. The value of this metric can not be calculated. By default, the value of the PFound metric is written to output data if YetiRank is optimized. This metric gives more accurate results on big datasets compared to YetiRank but it is significantly slower. This technique is described in the Winning The Transfer Learning Track of Yahoo!’s Learning To Rank Challenge with YetiRank paper. Note. The object weights are not used to optimize this metric. The group weights are used instead. This objective is used to optimize PairLogit. Automatically generated object pairs are used for this purpose. These pairs are generated independently for each object group. Use the Group weights file or the GroupWeight column of the Column descriptions file to change the group importance. In this case, the weight of each generated pair is multiplied by the value of the corresponding group weight.
QueryCrossEntropy	+	alpha Default: 0.95	Calculation principles $\text{[math]}$ See the QueryCrossEntropy section for more details.
QueryRMSE	+	use_weights Default: true	Calculation principles $\text{[math]}$
QuerySoftMax	+	use_weights Default: true	Calculation principles $\text{[math]}$
PFound *	–	decay Default: 0.85 top Default: –1 (all label values are used) use_weights Default: true	Calculation principles $\text{[math]}$ $\text{[math]}$ See the PFound section for more details.
NDCG *	_	top Default: –1 (all label values are used) use_weights Default: true type Default: Base	Calculation principles $\text{[math]}$ See the NDCG section for more details.
AverageGain	–	top Default: This parameter is obligatory (the default value is not defined) use_weights Default: true	Represents the average value of the label values for objects with the defined top $\text{[math]}$ label values. See the AverageGain section for more details.
PrecisionAt	–	top Default: –1 (all label values are used) border Default: 0.5	Calculation principles The calculation of this function consists of the following steps: The objects are sorted in descending order of predicted relevancies ( $\text{[math]}$ ) The metric is calculated as follows: $\text{[math]}$ $\text{[math]}$
RecallAt	–	top Default: –1 (all label values are used) border Default: 0.5	Calculation principles The calculation of this function consists of the following steps: The objects are sorted in descending order of predicted relevancies ( $\text{[math]}$ ) The metric is calculated as follows: $\text{[math]}$ $\text{[math]}$
MAP	–	top Default: –1 (all label values are used) border Default: 0.5	Calculation principles The objects are sorted in descending order of predicted relevancies ( $\text{[math]}$ ) The metric is calculated as follows: $\text{[math]}$ $\text{[math]}$ is the number of groups $\text{[math]}$ The value is calculated individually for each j-th group. $\text{[math]}$ $\text{[math]}$


Name	Used for optimization	User-defined parameters	Formula and/or description
YetiRank *	+	decay Default: 0.99 permutations Default: 10 use_weights Default: true	An approximation of ranking metrics (such as NDCG and PFound). Allows to use ranking metrics for optimization. The value of this metric can not be calculated. By default, the value of the PFound metric is written to output data if YetiRank is optimized. This metric gives less accurate results on big datasets compared to YetiRankPairwise but it is significantly faster. Note. The object weights are not used to optimize this metric. The group weights are used instead. This objective is used to optimize PairLogit. Automatically generated object pairs are used for this purpose. These pairs are generated independently for each object group. Use the Group weights file or the GroupWeight column of the Column descriptions file to change the group importance. In this case, the weight of each generated pair is multiplied by the value of the corresponding group weight.
YetiRankPairwise *	+	decay Default: 0.99 permutations Default: 10 use_weights Default: true	An approximation of ranking metrics (such as NDCG and PFound). Allows to use ranking metrics for optimization. The value of this metric can not be calculated. By default, the value of the PFound metric is written to output data if YetiRank is optimized. This metric gives more accurate results on big datasets compared to YetiRank but it is significantly slower. This technique is described in the Winning The Transfer Learning Track of Yahoo!’s Learning To Rank Challenge with YetiRank paper. Note. The object weights are not used to optimize this metric. The group weights are used instead. This objective is used to optimize PairLogit. Automatically generated object pairs are used for this purpose. These pairs are generated independently for each object group. Use the Group weights file or the GroupWeight column of the Column descriptions file to change the group importance. In this case, the weight of each generated pair is multiplied by the value of the corresponding group weight.
QueryCrossEntropy	+	alpha Default: 0.95	Calculation principles $\text{[math]}$ See the QueryCrossEntropy section for more details.
QueryRMSE	+	use_weights Default: true	Calculation principles $\text{[math]}$
QuerySoftMax	+	use_weights Default: true	Calculation principles $\text{[math]}$
PFound *	–	decay Default: 0.85 top Default: –1 (all label values are used) use_weights Default: true	Calculation principles $\text{[math]}$ $\text{[math]}$ See the PFound section for more details.
NDCG *	_	top Default: –1 (all label values are used) use_weights Default: true type Default: Base	Calculation principles $\text{[math]}$ See the NDCG section for more details.
AverageGain	–	top Default: This parameter is obligatory (the default value is not defined) use_weights Default: true	Represents the average value of the label values for objects with the defined top $\text{[math]}$ label values. See the AverageGain section for more details.
PrecisionAt	–	top Default: –1 (all label values are used) border Default: 0.5	Calculation principles The calculation of this function consists of the following steps: The objects are sorted in descending order of predicted relevancies ( $\text{[math]}$ ) The metric is calculated as follows: $\text{[math]}$ $\text{[math]}$
RecallAt	–	top Default: –1 (all label values are used) border Default: 0.5	Calculation principles The calculation of this function consists of the following steps: The objects are sorted in descending order of predicted relevancies ( $\text{[math]}$ ) The metric is calculated as follows: $\text{[math]}$ $\text{[math]}$
MAP	–	top Default: –1 (all label values are used) border Default: 0.5	Calculation principles The objects are sorted in descending order of predicted relevancies ( $\text{[math]}$ ) The metric is calculated as follows: $\text{[math]}$ $\text{[math]}$ is the number of groups $\text{[math]}$ The value is calculated individually for each j-th group. $\text{[math]}$ $\text{[math]}$