MLA-C01 Question 193
Select 2A data scientist is training a random forest model to predict customer churn using a dataset with 100,000 rows and 50 features. During the initial experiments, the model performs well on the training set but shows suboptimal performance on the validation set. The data scientist suspects the model is overfitting. Which hyperparameter adjustments could help mitigate overfitting?
- A
Increase the number of trees in the random forest
- B
Reduce the maximum depth of each tree
- C
Increase the minimum number of samples required to split a node
- D
Increase the number of features considered when splitting a node
- E
Enable bootstrapping to sample data with replacement
Show answer and explanation
Correct answers: B, C
Explanation
Overfitting in random forests often occurs when individual trees are too complex, capturing noise in the training data. Adjusting hyperparameters like reducing the maximum depth of trees or increasing the minimum samples required for splitting can constrain the complexity of the model and improve generalization. Increasing the number of trees or enabling bootstrapping addresses variance reduction, not overfitting.
- A. Incorrect.
Increasing the number of trees in a random forest generally improves performance by reducing variance, but it does not directly address overfitting caused by overly complex trees.
- B. Correct.
Reducing the maximum depth of each tree limits the complexity of the individual trees, preventing them from overfitting to the training data.
- C. Correct.
Increasing the minimum number of samples required to split a node reduces the likelihood of small splits, which can lead to overly specific tree structures that overfit the training data.
- D. Incorrect.
Increasing the number of features considered when splitting a node increases the complexity of the model and may exacerbate overfitting rather than mitigating it.
- E. Incorrect.
Bootstrapping is a standard feature of random forests and is already used to reduce variance across trees, but it does not directly address the issue of overfitting by individual trees.