Google Professional Machine Learning Engineer Question 207
Select 4Google Cloud PlatformYou are designing a generative AI application using Google Cloud. The application generates personalized marketing content for users. To evaluate the effectiveness of your generative AI solution, which of the following metrics and strategies should you prioritize?
- A
Use BLEU or ROUGE scores to measure the similarity between generated content and a reference dataset.
- B
Evaluate user engagement metrics, such as click-through rates or time spent on the generated content.
- C
Perform manual content reviews to assess the factual accuracy and appropriateness of generated content.
- D
Rely solely on the model's loss function during training as an indicator of its performance.
- E
Use diversity metrics to ensure the generated content is not repetitive and provides varied outputs for different inputs.
Show answer and explanation
Correct answers: A, B, C, E
Explanation
Evaluating generative AI solutions requires a combination of automated and manual evaluation techniques. BLEU or ROUGE scores are useful for assessing content quality, while user engagement metrics provide insight into the solution's effectiveness in real-world scenarios. Manual reviews ensure that the content meets business and ethical standards, and diversity metrics help confirm that the model produces varied outputs. Relying solely on the loss function is insufficient, as it does not reflect the model's performance in deployment or its ability to meet application-specific requirements.
- A. Correct.
BLEU or ROUGE scores are relevant for assessing the quality of text generation by comparing it to ground truth data, making them useful for evaluating generative AI solutions.
- B. Correct.
User engagement metrics help determine the real-world effectiveness of generated content, especially in applications like personalized marketing.
- C. Correct.
Manual content reviews are essential to ensure the generated content aligns with business goals, is factually accurate, and adheres to ethical standards.
- D. Incorrect.
The model's loss function is a useful indicator during training but is insufficient on its own to evaluate the real-world performance of generative AI solutions.
- E. Correct.
Diversity metrics are important to ensure the model generates varied and creative outputs, which is particularly crucial for user-facing applications like marketing.