MLA-C01 Question 189
Select 2You are training a gradient-boosted tree model using Amazon SageMaker, and you notice that the model's performance on the validation dataset is degrading despite improving accuracy on the training dataset. Which hyperparameter adjustment is MOST likely to help mitigate this issue?
- A
Decrease the learning rate of the model.
- B
Increase the maximum depth of the trees.
- C
Increase the number of trees in the model.
- D
Decrease the maximum depth of the trees.
Show answer and explanation
Correct answers: A, D
Explanation
The symptoms described indicate overfitting, where the model performs well on the training dataset but poorly on the validation dataset. Reducing the learning rate and decreasing the maximum depth of the trees are common strategies to reduce overfitting in gradient-boosted tree models. Both adjustments help control the model's complexity and improve its ability to generalize to unseen data.
- A. Correct.
Decreasing the learning rate slows down the model's training process, which can help prevent overfitting and improve generalization performance on the validation dataset.
- B. Incorrect.
Increasing the maximum depth of the trees can lead to overfitting, as deeper trees tend to capture more noise in the training data, worsening validation performance.
- C. Incorrect.
Increasing the number of trees may exacerbate overfitting if the model is already memorizing the training data and performing poorly on the validation dataset.
- D. Correct.
Decreasing the maximum depth of the trees reduces the model's complexity, which can help prevent overfitting and improve generalization performance on the validation dataset.