Google Professional Machine Learning Engineer Question 509
Select 3Google Cloud PlatformYou have deployed a machine learning model to production using Google Cloud AI Platform and are tasked with monitoring its performance over time. The business team indicates a decrease in model utility based on user feedback. To address this, you decide to compare the model's performance against a baseline model and a simpler heuristic-based model. Which actions should you take to effectively monitor and diagnose the issue?
- A
Set up continuous evaluation using the Vertex AI model monitoring feature to track prediction drift and feature skew.
- B
Compare the deployed model's performance metrics with a baseline model on the same evaluation dataset used during development.
- C
Deploy a simpler heuristic model to production and route 50% of traffic to it to compare live performance metrics.
- D
Analyze the model's performance over time by reviewing metrics such as accuracy, precision, and recall using Cloud Monitoring dashboards.
- E
Retrain the production model immediately with new data to address user feedback.
Show answer and explanation
Correct answers: A, B, D
Explanation
To effectively monitor and diagnose model performance issues, it is important to use tools like Vertex AI model monitoring to track data drift and feature skew, compare the current model's performance against a baseline, and analyze performance metrics over time. These steps help identify and diagnose potential issues without prematurely retraining the model, ensuring a systematic approach to maintaining model performance.
- A. Correct.
Correct: Vertex AI's model monitoring feature can track prediction drift and feature skew, which helps identify changes in data distribution or input features over time.
- B. Correct.
Correct: Comparing the deployed model with a baseline model on the same evaluation dataset allows you to assess if the deployed model is still performing as expected relative to a known standard.
- C. Incorrect.
Incorrect: Deploying a simpler heuristic model to production and splitting traffic is not a typical practice for monitoring. Instead, simpler models are used for offline comparisons or as baselines.
- D. Correct.
Correct: Reviewing metrics such as accuracy, precision, and recall over time can help detect performance degradation and provide insights into potential issues.
- E. Incorrect.
Incorrect: Retraining the model immediately without diagnosing the root cause of the performance drop may lead to suboptimal results and does not address monitoring requirements.