NCA-GENM Question 101
Select 3You are building a generative AI model that combines text and image data to generate captions for product images. During testing, you notice that the model's performance is inconsistent across different product categories. What should you do to ensure the quality and consistency of your test data in this multimodal setting?
- A
Ensure that the test dataset has an equal representation of all product categories.
- B
Verify that the text descriptions in the test dataset are free of grammatical errors and ambiguities.
- C
Exclude product categories that the model struggles with from the test dataset to improve evaluation metrics.
- D
Check if the images in the test dataset have consistent resolution and quality across all samples.
- E
Use synthetic text and image data to ensure the test dataset is larger, even if it lacks diversity.
Show answer and explanation
Correct answers: A, B, D
Explanation
In a multimodal setting, the quality and consistency of test data are crucial for accurately evaluating the model's performance. Ensuring equal representation of product categories, verifying clean and unambiguous text descriptions, and maintaining uniform image resolution and quality are key steps to test data preparation. Excluding categories or relying solely on synthetic data can compromise the evaluation process and provide misleading insights into the model's true capabilities.
- A. Correct.
Ensuring equal representation of product categories in the test dataset helps evaluate the model's performance fairly across all categories, making it a critical step in ensuring test data quality.
- B. Correct.
Grammatical errors or ambiguities in the text descriptions can confuse the model and lead to inconsistent performance. Verifying text quality ensures the model is tested on clean, reliable data.
- C. Incorrect.
Excluding challenging categories from the test dataset biases the evaluation process and gives an inaccurate representation of the model's capabilities, making this approach incorrect.
- D. Correct.
Consistent resolution and quality of images are essential to ensure the input data is uniform and does not introduce variability that could affect model performance.
- E. Incorrect.
While synthetic data can increase the dataset size, a lack of diversity in the data can negatively impact testing, making this option unsuitable for ensuring test data quality and consistency.