MLS-C01 Question 208
Select 3You are developing a machine learning model to predict customer churn using Amazon SageMaker's built-in XGBoost algorithm. During hyperparameter tuning, you notice that increasing the number of trees improves accuracy on the training data but eventually leads to overfitting on the validation set. Which of the following adjustments can help balance model performance and prevent overfitting?
- A
Reduce the maximum depth of each tree.
- B
Increase the number of trees further to improve generalization.
- C
Add regularization by tuning the
lambdaoralphahyperparameters. - D
Use early stopping to terminate training when validation performance stops improving.
- E
Increase the learning rate to converge faster.
Show answer and explanation
Correct answers: A, C, D
Explanation
Overfitting occurs when the model becomes too complex and starts modeling noise in the training data. To mitigate this, you can reduce the maximum tree depth to simplify the model, apply regularization (lambda or alpha) to penalize complexity, and use early stopping to halt training when validation performance no longer improves. Increasing the number of trees or learning rate would not alleviate overfitting in this scenario.
- A. Correct.
Reducing the maximum depth of each tree can decrease the model's complexity, which helps prevent overfitting by limiting how much the model can memorize the training data.
- B. Incorrect.
Increasing the number of trees further will likely exacerbate overfitting since the model is already showing signs of overfitting with the current number of trees.
- C. Correct.
Adding regularization through the
lambdaoralphahyperparameters helps control the size of leaf weights and penalizes overly complex models, thereby reducing overfitting. - D. Correct.
Using early stopping ensures training halts once the model's performance on the validation set stops improving, preventing the model from overfitting by training for too long.
- E. Incorrect.
Increasing the learning rate typically speeds up convergence but can make the model less stable and is unlikely to address overfitting directly.