MLS-C01 Question 212
Single answerYou are training a linear regression model using Amazon SageMaker. During the training process, you notice that the training loss is oscillating and not converging to a minimum. Which of the following is the MOST likely cause of this issue?
- A
The learning rate is too high.
- B
The dataset contains highly correlated features.
- C
The number of features in the dataset is too large.
- D
The training data lacks sufficient diversity.
Show answer and explanation
Correct answer: A
Explanation
The learning rate controls the size of the steps taken during the optimization process. If the learning rate is set too high, the model can overshoot the minimum of the loss function during updates, causing oscillations or divergence in the loss. To address this issue, you can decrease the learning rate to allow the model to converge more smoothly.
- A. Correct.
If the learning rate is too high, the model can overshoot the optimal solution during each update step, leading to oscillations in the loss function and preventing convergence.
- B. Incorrect.
Highly correlated features can cause multicollinearity, but this issue typically affects the interpretability and stability of the model, not oscillations in the training loss.
- C. Incorrect.
A large number of features can increase computational complexity and risk overfitting, but it does not directly cause oscillations in the loss function.
- D. Incorrect.
Lack of diversity in the training data can lead to poor generalization and overfitting, but it does not explain the oscillations in the training loss.