get_tokens

Return tokens that correspond to the given identifiers.

Method call format

get_tokens(token_id)

Parameters

token_id

Description

A list of token identifiers that should be returned.

Data types

list

Default value

Obligatory parameter

Type of return value

list

Example

from catboost.text_processing import Dictionary

dictionary = Dictionary(occurence_lower_bound=0)\
    .fit(["his", "tender", "heir", "whatever"])

print(dictionary.get_tokens([1,3]))

Output:

['his', 'whatever']