MLA-C01 Question 219
Select 2You are training a machine learning model on Amazon SageMaker to predict customer churn. During the evaluation phase, you notice that the model performs well on the training dataset but poorly on the validation dataset. Additionally, increasing the model complexity does not improve validation performance. Which of the following methods can help identify whether the model is overfitting or underfitting?
- A
Plot the training and validation learning curves to compare their performance trends.
- B
Use k-fold cross-validation to evaluate the model on multiple splits of the dataset.
- C
Check the model's performance on a held-out test dataset.
- D
Increase the model's complexity to observe whether the validation performance improves.
- E
Regularize the model using techniques like L1 or L2 regularization.
Show answer and explanation
Correct answers: A, B
Explanation
To effectively diagnose overfitting or underfitting in a machine learning model, analyzing the training and validation learning curves and using techniques like k-fold cross-validation provide actionable insights into the model's behavior. These methods help determine whether the model is performing poorly due to insufficient training or excessive complexity.
- A. Correct.
Plotting the training and validation learning curves provides insight into whether the model is overfitting (large gap between training and validation performance) or underfitting (both training and validation performance are poor).
- B. Correct.
Using k-fold cross-validation allows you to evaluate model performance on multiple splits of the dataset, helping identify patterns of overfitting or underfitting more robustly.
- C. Incorrect.
While checking performance on a held-out test dataset can give an idea of generalization, it does not directly help in diagnosing overfitting or underfitting during model training.
- D. Incorrect.
Increasing model complexity without additional diagnostics will not help identify whether the model is overfitting or underfitting. It could exacerbate overfitting instead.
- E. Incorrect.
Regularization is a technique to mitigate overfitting but does not help in diagnosing whether the model is overfitting or underfitting.