Databricks Machine Learning Associate Question 196
Single answerWhy can one-hot encoding categorical features be considered inefficient when training tree-based models such as decision trees or random forests?
- A
One-hot encoding leads to a significant increase in the dimensionality of the dataset, which can cause the tree-based model to become less interpretable.
- B
Tree-based models are inherently incapable of handling categorical features, so one-hot encoding is mandatory for their use.
- C
One-hot encoding introduces sparsity in the dataset, which can result in unnecessary computational overhead for tree-based model splits.
- D
Tree-based models naturally handle categorical features without requiring one-hot encoding, making it redundant in most cases.
Show answer and explanation
Correct answer: C
Explanation
One-hot encoding transforms a categorical feature with 'n' unique categories into 'n' binary features, introducing sparsity and increasing dataset dimensionality. While tree-based models are robust to categorical data, the sparsity and increased dimensionality from one-hot encoding can lead to unnecessary computational overhead when evaluating splits during training. Consequently, one-hot encoding is often inefficient for tree-based models.
- A. Incorrect.
While one-hot encoding does increase dimensionality, this does not directly impact the interpretability of tree-based models. The inefficiency lies more in computational overhead and sparsity introduced by one-hot encoding.
- B. Incorrect.
This statement is incorrect. Tree-based models, such as decision trees and random forests, can handle categorical features natively or through ordinal encoding without requiring one-hot encoding.
- C. Correct.
This is correct. One-hot encoding introduces many zero values (sparsity), which can result in unnecessary computational costs when tree-based models attempt to evaluate splits on the many generated features.
- D. Incorrect.
Although tree-based models can handle categorical features natively, this option does not explain why one-hot encoding is inefficient. Instead, it describes the flexibility of tree-based models.