MLS-C01 Question 214
Select 2You are training a linear regression model using Amazon SageMaker, and you notice that the model's loss is fluctuating widely during training. You suspect the learning rate might be contributing to the instability. Which of the following actions would help address this issue?
- A
Reduce the learning rate to a smaller value.
- B
Increase the learning rate to speed up convergence.
- C
Use learning rate scheduling to gradually decrease the learning rate over time.
- D
Switch to a non-linear model to improve stability.
- E
Increase the number of epochs to allow the model to stabilize.
Show answer and explanation
Correct answers: A, C
Explanation
When training a linear model, a high learning rate can cause instability in the training process by making excessively large updates to the model's weights. Reducing the learning rate ensures more controlled updates, while learning rate scheduling can further stabilize the training process by decreasing the learning rate over time as the model approaches convergence. Both of these approaches are effective in addressing the issue of fluctuating loss during training.
- A. Correct.
Reducing the learning rate can help stabilize the training process by ensuring smaller, more controlled updates to the model's weights, especially for linear models.
- B. Incorrect.
Increasing the learning rate could worsen the instability of the training process, causing the loss to diverge further.
- C. Correct.
Using learning rate scheduling to gradually decrease the learning rate over time helps maintain stability as the model approaches convergence, making it a suitable solution.
- D. Incorrect.
Switching to a non-linear model may not address the specific issue of learning rate instability for a linear model. The problem is related to the training process rather than the model type.
- E. Incorrect.
Increasing the number of epochs does not directly address the instability caused by a high learning rate. The issue lies in how updates are applied during training, not how many updates are made.