Databricks Machine Learning Associate Question 197
Select 3Why might one-hot encoding categorical features be inefficient when training tree-based models?
- A
One-hot encoding can lead to a significant increase in the dimensionality of the dataset, especially for high-cardinality categorical features.
- B
Tree-based models cannot handle categorical features and require all features to be one-hot encoded.
- C
One-hot encoded features can create sparse data representations, which may not align well with how splits are determined in tree-based models.
- D
Tree-based models can directly handle categorical features without the need for one-hot encoding in many cases.
- E
One-hot encoding reduces the interpretability of tree-based models by creating too many features.
Show answer and explanation
Correct answers: A, C, D
Explanation
One-hot encoding can lead to inefficiencies in tree-based models for several reasons. First, it increases the dimensionality of the dataset, especially when dealing with high-cardinality categorical features. This can make training slower and more resource-intensive. Additionally, the sparse nature of one-hot encoded data doesn't align well with the way tree-based models evaluate splits. Moreover, modern tree-based algorithms, like LightGBM and CatBoost, can directly handle categorical features efficiently without requiring one-hot encoding, making this transformation unnecessary in many cases.
- A. Correct.
Correct. One-hot encoding increases the number of features, especially for categorical variables with many unique values, which can lead to inefficiency in training tree-based models.
- B. Incorrect.
Incorrect. This statement is false, as tree-based models like decision trees and gradient-boosted trees can handle categorical features directly in some implementations (e.g., LightGBM, CatBoost) without needing one-hot encoding.
- C. Correct.
Correct. One-hot encoding creates sparse data, which can make it less efficient for tree-based models to determine splits, as the model has to evaluate many more potential splits.
- D. Correct.
Correct. Many modern tree-based algorithms, such as LightGBM and CatBoost, have built-in support for categorical features, making one-hot encoding unnecessary and inefficient in these cases.
- E. Incorrect.
Incorrect. While one-hot encoding increases the number of features, it does not inherently reduce the interpretability of tree-based models, which remain interpretable regardless of feature encoding.