---
metadata:
  - name: generator
    content: Diplodoc Platform v5.52.0
alternate:
  - https://catboost.ai/docs/en/references/text-processing__specification-example.md
  - href: en/references/text-processing__specification-example.md
    type: text/markdown
    title: Markdown version
  - href: ../llms.txt
    type: text/markdown
    title: llms.txt
---
> **Documentation Index:** Fetch the complete configuration index at https://catboost.ai/docs/en/llms.txt

# Text processing JSON specification example

```json
"text_processing_options" : {
    "tokenizers" : [{
        "tokenizer_id" : "Space",
        "delimiter" : " ",
        "lowercasing" : "true"
    }],

    "dictionaries" : [{
        "dictionary_id" : "BiGram",
        "gram_order" : "2"
    }, {
        "dictionary_id" : "Word",
        "gram_order" : "1"
    }],

    "feature_processing" : {
        "default" : [{
            "dictionaries_names" : ["Word"],
            "feature_calcers" : ["BoW"],
            "tokenizers_names" : ["Space"]
        }],

        "1" : [{
            "tokenizers_names" : ["Space"],
            "dictionaries_names" : ["BiGram", "Word"],
            "feature_calcers" : ["BoW"]
        }, {
            "tokenizers_names" : ["Space"],
            "dictionaries_names" : ["Word"],
            "feature_calcers" : ["NaiveBayes"]
        }]
    }
}
```

In this example:
- A single split-by-delimiter tokenizer is specified. It lowercases tokens after splitting.
- Two dictionaries: unigram (identified <q>Word</q>) and bigram (identified <q>BiGram</q>).
- Two feature calcers are specified for the second text feature:
    - BoW, which uses the <q>BiGram</q> and <q>Word</q> dictionaries.
    - NaiveBayes, which uses the <q>Word</q> dictionary.

- A single feature calcer is specified for all other text features: BoW, which uses the <q>Word</q> dictionary.