NDCG

This function is usually used to assess the quality of ranking.

• Calculation principles
• User-defined parameters

Calculation principles

The calculation of this function consists of the following steps:

1. The objects in each group are sorted in descending order of predicted relevancies ($a_{i}$).

2. The DCG metric is calculated for each group ($group \in groups$) with sorted objects (see step 1).

   The calculation principle depends on the specified value of the type and denominator parameters:

   type/denominator	LogPosition	Position
   Base	$DCG(group,top) = \sum\limits_{i=1}^{top}\displaystyle\frac{t_{g(i,group)}}{log_{2}(i+1)}$	$DCG(group,top) = \sum\limits_{i=1}^{top}\displaystyle\frac{t_{g(i,group)}}{i}$
   Exp	$DCG(group,top) = \sum\limits_{i=1}^{top}\displaystyle\frac{2^{t_{g(i,group)}} - 1}{log_{2}(i+1)}$	$DCG(group,top) = \sum\limits_{i=1}^{top}\displaystyle\frac{2^{t_{g(i,group)}} - 1}{i}$

   $t_{g(i,group)}$ is the label value for the i-th object in the group.

3. The objects in each group are sorted in descending order of target relevancies ($t_{i}$).

4. The iDCG metric is calculated for each group ($group \in groups$) with sorted objects (see step 3).

   The calculation principle depends on the specified value of the type and denominator parameters:

   type/denominator	LogPosition	Position
   Base	$IDCG(group,top) = \sum\limits_{i=1}^{top}\displaystyle\frac{t_{g(i,group)}}{log_{2}(i+1)}$	$IDCG(group,top) = \sum\limits_{i=1}^{top}\displaystyle\frac{t_{g(i,group)}}{i}$
   Exp	$IDCG(group,top) = \sum\limits_{i=1}^{top}\displaystyle\frac{2^{t_{g(i,group)}} - 1}{log_{2}(i+1)}$	$IDCG(group,top) = \sum\limits_{i=1}^{top}\displaystyle\frac{2^{t_{g(i,group)}} - 1}{i}$

5. The NDCG metric is calculated for each group:
   $nDCG(group,top) = \displaystyle\frac{DCG}{iDCG}$

6. The aggregated value of the metric for all groups is calculated as follows:
   $nDCG(top) = \frac{\sum\limits_{group \in groups} nDCG(group, top) * w_{group}}{\sum\limits_{group \in groups} w_{group}}$

User-defined parameters

top

Description

The number of top samples in a group that are used to calculate the ranking metric. Top samples are either the samples with the largest approx values or the ones with the lowest target values if approx values are the same.

Default: –1 (all label values are used)

use_weights

Description

Use object/group weights to calculate metrics if the specified value is true and set all weights to 1 regardless of the input data if the specified value is false.

Default: true

type

Description

Metric calculation principles.

Possible values:

• Base
• Exp

Default: Base

denominator

Description

Metric denominator type.

Possible values:

• LogPosition
• Position

Default: LogPosition