CatBoost provides the following model analysis tools:
Feature importance
ShapValues
Feature analysis charts
Feature interaction
Object importance