NCA-GENM Question 7
Select 3During the training of a multimodal generative AI model, you observe that the model's loss oscillates significantly, leading to unstable training. Which of the following strategies can help stabilize the training process?
- A
Use gradient clipping to prevent exploding gradients.
- B
Increase the learning rate to force faster convergence.
- C
Normalize the input data for each modality.
- D
Implement a curriculum learning approach to gradually introduce more complex data.
- E
Reduce the batch size to decrease memory usage.
Show answer and explanation
Correct answers: A, C, D
Explanation
Stabilizing training in multimodal settings often involves addressing issues like gradient instability, imbalanced contributions from modalities, and overly complex data. Gradient clipping (Option 1) prevents exploding gradients, normalizing input data (Option 3) ensures balanced contributions from modalities, and curriculum learning (Option 4) reduces initial complexity to allow the model to stabilize before encountering more challenging data. These strategies directly contribute to training stability, unlike increasing the learning rate or reducing batch size, which may not address the root causes of instability.
- A. Correct.
Gradient clipping helps to stabilize training by capping the gradients at a defined threshold, preventing exploding gradients which can derail training.
- B. Incorrect.
Increasing the learning rate often exacerbates instability, as larger updates can lead to further oscillations or divergence.
- C. Correct.
Normalizing input data ensures that each modality contributes proportionally, reducing imbalances that can lead to unstable training.
- D. Correct.
Curriculum learning introduces simpler data initially, allowing the model to learn foundational patterns before tackling more complex data, which helps stabilize training.
- E. Incorrect.
Reducing the batch size primarily impacts memory usage rather than directly addressing stability issues, and in some cases, it can increase noise in gradient updates.