MLA-C01 Question 12
Select 3You are designing a machine learning solution for predicting customer churn using Amazon SageMaker. After training your model, you need to evaluate its performance and identify whether it is overfitting. Which combination of methods should you use to achieve this?
- A
Compare the training loss and validation loss curves during training
- B
Evaluate the model using a test dataset that was not seen during training
- C
Monitor the model's accuracy on the training dataset only
- D
Use k-fold cross-validation to assess model performance
- E
Increase the number of epochs and re-train the model to improve performance
Show answer and explanation
Correct answers: A, B, D
Explanation
To identify overfitting and evaluate the performance of a machine learning model, it is crucial to compare training and validation losses, use a test dataset for unbiased evaluation, and employ techniques like k-fold cross-validation. These methods provide insights into how well the model generalizes to unseen data, which is critical for avoiding overfitting.
- A. Correct.
Comparing the training loss and validation loss curves helps to identify overfitting. If the training loss is low but the validation loss is high, it may indicate overfitting.
- B. Correct.
Evaluating the model on a test dataset that was not seen during training provides an unbiased measure of model performance and helps validate its generalization capability.
- C. Incorrect.
Monitoring the model’s accuracy on the training dataset alone does not provide insights into overfitting, as it only reflects how well the model memorized the training data.
- D. Correct.
K-fold cross-validation is a robust method to evaluate model performance and detect overfitting by testing the model on multiple subsets of the data.
- E. Incorrect.
Increasing the number of epochs and re-training may exacerbate overfitting rather than addressing it, as the model could further memorize the training data.