NCA-GENM Question 5
Select 3During the training of a multimodal generative AI model, you observe frequent gradients exploding in the text encoder, leading to instability in the training process. Which of the following techniques can help control the stability of training in a multimodal setting?
- A
Apply gradient clipping to limit the magnitude of gradients during backpropagation
- B
Increase the batch size to reduce noise in gradient updates
- C
Use learning rate scheduling to dynamically adjust the learning rate during training
- D
Regularize the model by applying dropout to both text and image modalities
- E
Add layer normalization within the architecture for both modalities
Show answer and explanation
Correct answers: A, C, E
Explanation
Training stability in multimodal settings often requires techniques that directly address gradient-related issues, such as explosion or vanishing gradients. Gradient clipping (Option 1) ensures gradients remain within a manageable range. Learning rate scheduling (Option 3) prevents overly aggressive updates that could destabilize training. Layer normalization (Option 5) improves stability by standardizing activations, reducing the risk of gradients exploding or vanishing. While other options like increasing batch size or applying dropout may have benefits, they do not directly solve gradient explosion or training stability issues in multimodal contexts.
- A. Correct.
Gradient clipping is a key technique to control exploding gradients by capping their magnitude, ensuring stable training, especially in multimodal and deep architectures.
- B. Incorrect.
Increasing the batch size can reduce noise in gradient updates but does not directly address gradient explosion issues, making it less effective for this scenario.
- C. Correct.
Learning rate scheduling dynamically adjusts the learning rate, helping to stabilize training by preventing large updates that could lead to instability.
- D. Incorrect.
Dropout helps reduce overfitting by randomly deactivating neurons during training but does not directly address gradient explosion or stability issues.
- E. Correct.
Layer normalization standardizes activations within layers, reducing the risk of gradient explosion and improving training stability, particularly in complex multimodal models.