MLS-C01 Question 205
Select 2You are training a deep neural network model for image classification on AWS SageMaker. During training, you observe that the model's loss decreases very slowly, and the model seems to converge to suboptimal results. Upon reviewing the training configuration, you suspect that the issue is related to the learning rate and activation functions used. Which of the following actions can help address this problem?
- A
Increase the learning rate to accelerate convergence.
- B
Switch the activation function in hidden layers to ReLU.
- C
Decrease the learning rate to avoid overshooting the optimal solution.
- D
Use a sigmoid activation function in the output layer for a multi-class classification problem.
- E
Enable learning rate scheduling to dynamically adjust the learning rate during training.
- F
Add more layers to the neural network to improve its capacity.
Show answer and explanation
Correct answers: B, E
Explanation
In this scenario, slow convergence and suboptimal results are likely related to the choice of activation functions and the learning rate configuration. Switching to ReLU activation functions can help mitigate the vanishing gradient problem, thereby improving convergence speed. Enabling learning rate scheduling dynamically adjusts the learning rate, which helps optimize the training process without overshooting or stagnation. These two solutions directly address the observed issues during training.
- A. Incorrect.
Increasing the learning rate may lead to instability in training and cause the model to overshoot the optimal solution. This is not a recommended action to improve convergence in this scenario.
- B. Correct.
Switching to the ReLU activation function is a common approach to address slow convergence in deep neural networks because it helps mitigate the vanishing gradient problem often encountered with other activation functions like sigmoid or tanh.
- C. Incorrect.
Decreasing the learning rate can slow down training even further. While it may help avoid overshooting in some cases, it does not directly address the issue of slow convergence.
- D. Incorrect.
The sigmoid activation function is not suitable for a multi-class classification problem. For such tasks, using the softmax activation function in the output layer is more appropriate.
- E. Correct.
Enabling learning rate scheduling allows the learning rate to adjust dynamically, which helps improve convergence by starting with a higher learning rate and gradually reducing it as the model approaches the optimal solution.
- F. Incorrect.
Adding more layers might increase model capacity, but it does not directly address the issue of slow convergence and could lead to overfitting if not managed properly.