MLS-C01 Question 161
Select 3You are training a machine learning model on AWS SageMaker using a custom training script. During training, you notice that the loss function value decreases initially but plateaus after a few epochs, preventing the model from achieving optimal performance. Which of the following actions could help address this issue?
- A
Decrease the learning rate of the optimizer used in your training script.
- B
Switch the optimizer to one that uses momentum, such as Adam or SGD with momentum.
- C
Change the loss function to a simpler one, such as Mean Absolute Error (MAE).
- D
Adjust the batch size to ensure the model generalizes better during training.
- E
Implement learning rate scheduling to dynamically adjust the learning rate during training.
Show answer and explanation
Correct answers: A, B, E
Explanation
Plateauing loss during training often indicates an optimization issue. Decreasing the learning rate, using optimizers with momentum, and introducing learning rate scheduling are effective techniques to improve convergence and escape plateaus. These techniques adjust how the model updates weights, enabling better performance. While batch size and loss function choice are important considerations, they are not directly related to solving the plateauing issue in this scenario.
- A. Correct.
Decreasing the learning rate could help if the optimizer is overshooting the minima or settling in a suboptimal region. A smaller learning rate allows the optimizer to make smaller, more precise updates.
- B. Correct.
Optimizers like Adam or SGD with momentum can help escape plateaus or local minima by leveraging momentum to accelerate convergence.
- C. Incorrect.
Changing the loss function might not directly address the plateauing issue. The problem is likely related to optimization rather than the choice of the loss function.
- D. Incorrect.
Adjusting the batch size is typically more related to generalization or stability but is not a direct solution for a plateauing loss during training.
- E. Correct.
Learning rate scheduling dynamically adjusts the learning rate based on the training progress. This can help the model converge by reducing the learning rate as the loss plateaus.