MLS-C01 Question 207
Select 2You are training a deep neural network for an image classification task on Amazon SageMaker. During experimentation, you observe that the loss does not decrease significantly after the first few epochs, and the model's accuracy remains low. Which of the following actions could address this issue?
- A
Increase the learning rate to speed up convergence.
- B
Use a different activation function, such as ReLU instead of Sigmoid, in the hidden layers.
- C
Add more layers to the network to increase its representational capacity.
- D
Normalize the input data to have zero mean and unit variance.
- E
Reduce the batch size to make updates more frequent.
Show answer and explanation
Correct answers: B, D
Explanation
When a neural network's loss does not decrease significantly, it can often be due to issues like improper activation functions or poorly normalized input data. ReLU activation is generally more effective for hidden layers in deep networks due to its ability to mitigate the vanishing gradient problem. Additionally, normalizing input data helps the model converge faster and more effectively. Increasing the learning rate, adding more layers, or reducing the batch size may not directly address the issue and could lead to other complications.
- A. Incorrect.
Increasing the learning rate might lead to instability in training and could prevent the model from converging properly, especially if the learning rate is already appropriately set.
- B. Correct.
Using a different activation function like ReLU can help alleviate the vanishing gradient problem, which is common with Sigmoid activation, thereby improving training performance.
- C. Incorrect.
Adding more layers increases model complexity but does not guarantee better performance. It may exacerbate issues like overfitting or vanishing gradients if not handled properly.
- D. Correct.
Normalizing the input data ensures that all features are on a similar scale, which can improve convergence and model performance, especially in deep learning.
- E. Incorrect.
Reducing the batch size can make updates more frequent, but it does not directly address the underlying problem of poor convergence or accuracy in this scenario.