MLA-C01 Question 229
Select 2You are training a deep learning model using Amazon SageMaker, and you notice that the model's loss is not decreasing despite multiple epochs of training. Which of the following adjustments could help address the convergence issue? (Select TWO.)
- A
Decrease the learning rate to avoid overshooting the optimal solution.
- B
Increase the batch size to allow the model to converge faster.
- C
Normalize the input data to ensure features are on similar scales.
- D
Switch to a less complex optimizer, such as Stochastic Gradient Descent (SGD).
- E
Add dropout layers to counteract overfitting during training.
Show answer and explanation
Correct answers: A, C
Explanation
Convergence issues occur when the model's loss does not decrease during training. Decreasing the learning rate can stabilize the optimizer's updates, avoiding oscillations or overshooting the minimum. Normalizing the input data ensures the optimizer can effectively navigate the loss landscape, as feature scaling significantly impacts gradient-based methods. Other options, such as increasing batch size, switching optimizers, or adding dropout, do not directly address convergence issues and are more relevant to other aspects of training like speed, stability, or overfitting.
- A. Correct.
Decreasing the learning rate can help the model converge better, as a high learning rate might be causing the optimizer to oscillate and overshoot the optimal solution.
- B. Incorrect.
Increasing the batch size does not directly address convergence issues. It primarily impacts training speed and stability but does not guarantee improvement in loss reduction.
- C. Correct.
Normalizing the input data ensures that features are on similar scales, which can help the optimizer converge more effectively by preventing one feature from dominating the gradient updates.
- D. Incorrect.
Switching to a simpler optimizer like SGD is not guaranteed to solve convergence issues. More advanced optimizers like Adam are generally better at handling such problems.
- E. Incorrect.
Adding dropout layers is primarily a regularization technique to reduce overfitting. It does not directly address convergence problems related to loss not decreasing.