Databricks Machine Learning Associate Question 194
Single answerWhy is one-hot encoding categorical features often inefficient for tree-based models?
- A
One-hot encoding produces sparse data, which increases the memory usage and computational cost for tree-based models.
- B
Tree-based models cannot handle categorical features encoded as one-hot vectors.
- C
One-hot encoding introduces multicollinearity, which negatively impacts tree-based models.
- D
Tree-based models can inherently handle categorical features without the need for one-hot encoding.
Show answer and explanation
Correct answer: A
Explanation
One-hot encoding is inefficient for tree-based models because it results in sparse matrices, which increase computational costs and memory usage. Tree-based models are capable of handling categorical features directly or with simpler encodings like label encoding. Thus, one-hot encoding is often unnecessary for these models and can lead to inefficiencies.
- A. Correct.
One-hot encoding creates sparse data matrices, especially when the categorical variable has many unique categories. This can increase memory and computational requirements unnecessarily for tree-based models, which are naturally robust to categorical features.
- B. Incorrect.
Tree-based models like decision trees, random forests, and gradient-boosted trees can handle one-hot encoded features, but their efficiency decreases due to data sparsity. This statement is incorrect because they can process one-hot encoded features, just not optimally.
- C. Incorrect.
Multicollinearity is not an issue for tree-based models because they are not sensitive to correlated features, unlike linear models. This makes the statement incorrect.
- D. Incorrect.
Tree-based models can inherently handle categorical features when they are properly encoded (e.g., label encoding or categorical splits), but this is not the main reason one-hot encoding is inefficient. This statement is partially true but does not address inefficiency caused by sparsity.