• Overview
  • Parameter tuning
  • Speeding up the training
  • FAQ
  • Development and contributions
  • Contacts

PFound

This function is usually used to assess the quality of ranking.

Calculation principles

The possible label values are limited: ti[0;1]t_{i} \in [0;1].

The calculation of this function consists of the following steps:

  1. The objects in each group are sorted in descending order of predicted relevancies (aia_{i}).

  2. The PFound metric is calculated for each group (groupgroupsgroup \in groups). To do this, the label values (tit_{i}) of the objects from the sorted list are multiplied by their weight (P(i,group,decay)P(i, group, decay)) and summed up as follows:

    PFound(group,top,decay)=i=0topP(i,group,decay)tg(i,group),wherePFound(group, top, decay) = \sum_{i = 0}^{top} P(i, group, decay) * t_{g(i, group)}{, where}

    • P(0,group,decay)=1P(0, group, decay) = 1
    • P(i,group,decay)=P(i1,group,decay)(1tg(i1,group))decayP(i, group, decay) = P(i – 1, group, decay) * (1 - t_{g(i – 1, group)}) * decay
    • decaydecay is a constant, decay[0,1]decay \in [0, 1]
    • g(i,group)g(i, group) is the global index of the ii-th best object in the group . The ii-th object is considered better than the jj-th object if the following inequality is true: ai>aja_{i} > a_{j}
  3. The aggregated value of the metric for all groups is calculated as follows:

    PFound(top,decay)=1groupgroupswgroupgroupgroupswgroupPFound(group,top,decay)PFound(top, decay) = \displaystyle\frac{1}{\sum\limits_{group\in groups} w_{group}}\sum_{group\in groups} w_{group} PFound(group, top, decay)

User-defined parameters

decay

Description

The probability of search continuation after reaching the current object.

top

The number of top samples in a group that are used to calculate the ranking metric. Top samples are either the samples with the largest approx values or the ones with the lowest target values if approx values are the same.

Default: –1 (all label values are used)