Google Professional Machine Learning Engineer Question 204
Select 3Google Cloud PlatformYou are a Machine Learning Engineer tasked with evaluating a text-to-image generative AI model that your team has trained. The model generates images based on textual prompts. Which of the following actions should you prioritize to effectively evaluate the model's performance?
- A
Measure the model's BLEU score to evaluate the quality of the generated images based on the textual prompts.
- B
Conduct user evaluations to assess how well the generated images align with user expectations and the given prompts.
- C
Evaluate the diversity of the generated images for a single prompt to ensure the model avoids mode collapse.
- D
Use perceptual similarity metrics, such as LPIPS, to compare generated images with ground truth or reference images.
- E
Perform a latency test to measure the time it takes for the model to generate an image for a given prompt.
Show answer and explanation
Correct answers: B, C, D
Explanation
Evaluating generative AI solutions like text-to-image models requires a multifaceted approach. User evaluations, diversity assessment, and perceptual similarity metrics are critical for understanding the alignment, quality, and variety of the generated outputs. BLEU scores and latency tests are not directly relevant to assessing the quality of image generation in this context.
- A. Incorrect.
BLEU score is primarily used for evaluating text generation tasks, such as machine translation or summarization. It is not applicable for evaluating the quality of images generated by a text-to-image model.
- B. Correct.
User evaluations are crucial for understanding how well the model meets user expectations and aligns with the intent of the textual prompts. This is an important aspect of evaluating generative AI solutions.
- C. Correct.
Evaluating the diversity of the generated images is vital to ensure the model does not repeatedly generate similar outputs, which could indicate a problem like mode collapse.
- D. Correct.
Perceptual similarity metrics, such as LPIPS, help quantify how similar the generated images are to reference or ground truth images. This is a standard approach for evaluating generative models.
- E. Incorrect.
While latency is important from an operational perspective, it is not a direct measure of how well the model generates images based on textual prompts.