NCA-GENM Question 9
Select 3During the training of a multimodal generative AI model, you observe that the training loss for one modality (e.g., vision) is decreasing consistently, while the loss for another modality (e.g., text) fluctuates significantly. Which approaches could help stabilize the training process across modalities?
- A
Use modality-specific learning rates to fine-tune the contribution of each modality during updates.
- B
Normalize the input data for each modality to ensure consistent feature scaling.
- C
Increase the batch size to improve the convergence of the training process.
- D
Implement gradient clipping to limit excessive updates for unstable modalities.
- E
Remove the modality with fluctuating loss to simplify the training process.
Show answer and explanation
Correct answers: A, B, D
Explanation
Stabilizing training in multimodal settings often requires addressing imbalances between modalities. Techniques such as modality-specific learning rates, input normalization, and gradient clipping directly target the factors causing instability, ensuring that each modality contributes effectively to the overall training process. Simply increasing batch size or removing a modality does not address the root cause and could negatively impact the model's performance and purpose.
- A. Correct.
Using modality-specific learning rates can help balance the learning process for each modality, especially if one modality dominates or lags behind during training.
- B. Correct.
Normalizing input data ensures that features from different modalities are scaled appropriately, reducing potential disparities in their contributions to the loss.
- C. Incorrect.
Increasing the batch size can improve stability in some cases, but it does not address the core issue of modality imbalance and may lead to resource constraints.
- D. Correct.
Gradient clipping is an effective method to address instability caused by large updates, particularly in modalities with fluctuating losses.
- E. Incorrect.
Removing a modality is not a recommended approach, as it defeats the purpose of training a multimodal model and reduces its overall capability.