Tokenizer options

    The following is a list of options for the --tokenizers parameter (these options are set in option_name):

    lowercasing

    Description

    Convert tokens to lower case.

    Data types bool

    Default value Tokens are not converted to lower case