MLS-C01 Question 164
Select 3You are training a deep learning model using batch gradient descent on AWS SageMaker. During training, you notice the loss function does not decrease significantly after multiple epochs, and the model seems to converge to a suboptimal solution. Which of the following techniques could help improve the convergence of your training process?
- A
Using a learning rate scheduler to adjust the learning rate dynamically during training
- B
Switching from batch gradient descent to stochastic gradient descent (SGD)
- C
Re-initializing the model weights with random values
- D
Using a more complex loss function to encourage better optimization
- E
Normalizing the input features to have zero mean and unit variance
Show answer and explanation
Correct answers: A, B, E
Explanation
Convergence issues during training can often be addressed by improving the optimization process. Techniques such as using a learning rate scheduler, switching to stochastic gradient descent, and normalizing input features improve the efficiency and effectiveness of gradient descent. Re-initializing weights or complicating the loss function do not directly address convergence problems and may introduce additional challenges.
- A. Correct.
Using a learning rate scheduler can dynamically adjust the learning rate during training, allowing the model to converge more effectively by reducing the learning rate when approaching a minimum.
- B. Correct.
Switching to stochastic gradient descent (SGD) can help the model escape suboptimal local minima or saddle points, as SGD introduces randomness in the gradient updates.
- C. Incorrect.
Re-initializing model weights randomly may not address the underlying issue and could lead to the same convergence problems unless other optimizations are applied.
- D. Incorrect.
Using a more complex loss function is not guaranteed to improve convergence and may make the optimization process more difficult.
- E. Correct.
Normalizing input features helps gradient descent converge faster by ensuring that the gradients are well-scaled and the optimization landscape is smoother.