NCA-GENM Question 105
Select 3You are developing a multimodal generative AI model that generates image captions based on provided images. After training the model, you must test its accuracy and effectiveness. Which of the following steps are most appropriate during the testing phase?
- A
Evaluate the model's performance using BLEU or ROUGE scores for text generation.
- B
Test the model with unseen images and compare generated captions to ground truth captions.
- C
Use only the training dataset for testing to ensure consistent results.
- D
Manually inspect a subset of generated captions for coherence and relevance.
- E
Focus exclusively on the model's computational efficiency rather than accuracy.
Show answer and explanation
Correct answers: A, B, D
Explanation
Testing a generative AI model's accuracy and effectiveness involves both quantitative and qualitative evaluation. Using metrics like BLEU or ROUGE quantifies performance, while testing with unseen data ensures the model generalizes well. Manual inspection provides additional insights into the model's coherence and relevance, which automated metrics may not fully capture.
- A. Correct.
Correct: BLEU and ROUGE are standard metrics for evaluating the quality of text generation, which is relevant in assessing the effectiveness of the captions generated by the model.
- B. Correct.
Correct: Testing with unseen images and comparing generated captions to ground truth is essential to evaluate the model's generalization capability and accuracy.
- C. Incorrect.
Incorrect: Using only the training dataset for testing undermines the purpose of testing, as it does not evaluate the model's ability to generalize to new data.
- D. Correct.
Correct: Manual inspection of generated outputs helps identify issues that automated metrics might miss, such as contextual relevance and coherence.
- E. Incorrect.
Incorrect: While computational efficiency is important, it should not be prioritized over accuracy and relevance during the testing phase of a generative AI model.