MLS-C01 Question 201
Single answerYou are tasked with training a deep learning model for image classification using Amazon SageMaker. To ensure the model initializes properly, you want to use a state-of-the-art weight initialization strategy to prevent vanishing or exploding gradients during training. Which of the following initialization strategies would be most appropriate for this scenario?
- A
Random initialization with values between 0 and 1
- B
Xavier (Glorot) initialization
- C
Constant initialization with all weights set to zero
- D
He initialization
Show answer and explanation
Correct answer: B
Explanation
Xavier (Glorot) initialization is a widely used weight initialization strategy for deep learning models, particularly when training image classification models. It ensures that the weights are initialized in a way that maintains the variance of activations across layers, reducing the risk of vanishing or exploding gradients. This makes it a preferred choice in scenarios like the one described in the question.
- A. Incorrect.
Random initialization with values between 0 and 1 is not ideal as it can lead to vanishing or exploding gradients, especially in deep networks. It lacks the mathematical foundation to ensure gradients flow properly through the network.
- B. Correct.
Xavier (Glorot) initialization is specifically designed to balance the variance of weights across layers in deep networks. It helps to prevent vanishing or exploding gradients, making it a suitable choice for initializing weights in image classification models.
- C. Incorrect.
Constant initialization with all weights set to zero is not recommended as it causes symmetry problems during training, leading to ineffective model learning.
- D. Incorrect.
He initialization is optimized for activation functions like ReLU and its variants. While it's effective for certain tasks, Xavier initialization is generally more suitable for image classification tasks where the activation functions might not always be ReLU-based.