Google Professional Machine Learning Engineer Question 508
Select 3Google Cloud PlatformYou are deploying a machine learning model for predicting customer churn. To ensure the model remains effective over time, you want to monitor its performance against a baseline model, a simpler logistic regression model, and its performance over the last six months. Which monitoring strategies should you implement to achieve this goal?
- A
Track the model's key metrics such as precision, recall, and AUC against the baseline model's metrics.
- B
Monitor the model's performance over time for drift by comparing recent predictions with historical predictions.
- C
Compare the model's latency and resource utilization with the simpler logistic regression model.
- D
Set up alerts for significant deviations in model metrics compared to the simpler model and the baseline.
- E
Periodically retrain the model even if no significant performance drop is observed.
Show answer and explanation
Correct answers: A, B, D
Explanation
Monitoring a model's performance over time involves evaluating it against baselines, simpler models, and historical trends. Tracking key metrics (e.g., precision, recall) relative to baselines provides a benchmark for effectiveness, while monitoring for drift ensures the model adapts to changing data. Alerts help identify performance issues quickly. Comparing latency or retraining without evidence of drift are not directly related to performance monitoring.
- A. Correct.
Correct. Comparing the model's precision, recall, and AUC against the baseline helps evaluate its effectiveness relative to a benchmark.
- B. Correct.
Correct. Monitoring for drift by comparing recent predictions with historical predictions is essential for ensuring the model's relevance over time.
- C. Incorrect.
Incorrect. While monitoring latency and resource utilization can be useful, this does not directly assess performance against baselines or over time.
- D. Correct.
Correct. Setting up alerts for deviations in metrics ensures timely detection of performance issues compared to simpler models and baselines.
- E. Incorrect.
Incorrect. Retraining a model without observing a performance drop is not efficient or necessary unless there is evidence of drift or outdated patterns.