MLS-C01 Question 215
Single answerYou are training a linear regression model using the AWS SageMaker Linear Learner algorithm. During training, you notice that the model's loss does not decrease and the model struggles to converge to an optimal solution. Which of the following could be a potential reason?
- A
The learning rate is too high, causing the model to overshoot the optimal solution.
- B
The dataset used for training has missing values.
- C
The learning rate is too low, resulting in the model taking too long to converge.
- D
The model is overfitting due to excessive training epochs.
Show answer and explanation
Correct answer: A
Explanation
The learning rate is a critical hyperparameter in training machine learning models. If the learning rate is too high, the model may fail to converge as it overshoots the optimal solution during gradient descent steps. This can result in the loss oscillating or diverging entirely, preventing the model from reducing its loss during training.
- A. Correct.
If the learning rate is too high, the model might oscillate around the optimal solution or diverge completely, causing the loss to not decrease.
- B. Incorrect.
While missing values in the dataset can cause issues, SageMaker typically requires data preprocessing to handle missing values before training starts. This is less likely to be the cause of the issue described.
- C. Incorrect.
A low learning rate would cause the model to converge slowly, but it would still result in a reduction in loss over time. The issue in this scenario is that the loss is not decreasing at all.
- D. Incorrect.
Overfitting is a separate issue that typically results in low training loss but poor performance on validation data. It does not explain why the loss fails to decrease during training.