MLA-C01 Question 233
Select 3You are training a deep learning model using Amazon SageMaker, but notice that the loss function does not decrease over multiple epochs. Which of the following could be the cause of this convergence issue?
- A
The learning rate is too high, causing the optimizer to overshoot the optimal solution.
- B
The model is underfitting due to insufficient model complexity or insufficient number of parameters.
- C
The dataset is imbalanced, leading to poor gradient updates.
- D
The activation function used in the model is causing vanishing gradients.
- E
The training data has been preprocessed incorrectly, such as normalization not being applied.
Show answer and explanation
Correct answers: A, D, E
Explanation
Convergence issues can arise due to various factors. A learning rate that is too high can cause the optimizer to overshoot the optimal solution, while vanishing gradients and incorrect data preprocessing can also hinder the optimization process. While model complexity and data imbalance affect performance, they are not direct causes of convergence issues.
- A. Correct.
A learning rate that is too high can cause the optimizer to oscillate or overshoot the optimal solution, preventing convergence. This is a common cause of convergence issues.
- B. Incorrect.
Underfitting typically results in poor performance but is not directly tied to convergence issues. The loss may still decrease even if the model is too simple.
- C. Incorrect.
An imbalanced dataset can lead to biased model predictions, but it does not directly cause convergence issues. It is more of a general performance issue.
- D. Correct.
Vanishing gradients, often caused by poorly chosen activation functions such as sigmoid in deep networks, can prevent the model from learning effectively, resulting in convergence problems.
- E. Correct.
Incorrect preprocessing, such as failing to normalize features, can lead to poor scaling of input data, making it difficult for the optimizer to converge.