Tokenizer options
The following is a list of options for the --tokenizers parameter (these options are set in option_name):
lowercasing
Description
Convert tokens to lower case.
Data types bool
Default value Tokens are not converted to lower case