MLS-C01 Question 163
Select 3A data scientist is training a deep learning model on Amazon SageMaker. During training, they observe that the validation loss stops improving after a few epochs, while the training loss continues to decrease. Which techniques could help address this issue?
- A
Implement early stopping by monitoring the validation loss.
- B
Increase the learning rate to make the model converge faster.
- C
Introduce dropout layers to reduce overfitting.
- D
Use a more complex model architecture to capture additional patterns in the data.
- E
Perform hyperparameter tuning to optimize regularization parameters.
Show answer and explanation
Correct answers: A, C, E
Explanation
The scenario describes a classic case of overfitting, where the model performs well on the training data but poorly on the validation data. Techniques like early stopping, introducing dropout layers, and hyperparameter tuning are effective in addressing overfitting. Increasing the learning rate or using a more complex model would not resolve the issue and might make it worse.
- A. Correct.
Early stopping is a common technique to prevent overfitting by halting training when the validation loss stops improving. This directly addresses the issue described.
- B. Incorrect.
Increasing the learning rate could lead to instability in training and is unlikely to resolve the issue of overfitting.
- C. Correct.
Dropout layers are used to reduce overfitting by randomly deactivating neurons during training, which helps improve generalization.
- D. Incorrect.
Using a more complex model architecture could exacerbate overfitting because the model may learn noise in the training data rather than generalizable patterns.
- E. Correct.
Hyperparameter tuning can help identify optimal regularization parameters, such as L1 or L2 regularization, which can mitigate overfitting.