Google Professional Machine Learning Engineer Question 492
Select 2Google Cloud PlatformYou are a Machine Learning Engineer deploying a model for predicting customer churn using Vertex AI. The model's predictions may degrade over time due to changes in customer behavior. To ensure the model remains reliable, you want to establish continuous evaluation and monitoring. Which approaches should you implement to monitor and evaluate the model effectively?
- A
Enable Vertex AI Model Monitoring to track prediction drift and feature skew.
- B
Set up a pipeline to retrain the model automatically whenever the training data is updated.
- C
Manually download prediction logs weekly and analyze them for anomalies.
- D
Use Vertex AI Model Monitoring to track model accuracy by comparing predictions with ground truth labels.
- E
Implement a batch inference job to monitor the model's latency periodically.
Show answer and explanation
Correct answers: A, D
Explanation
Continuous evaluation involves monitoring the model's prediction quality and input data to ensure it remains reliable over time. Vertex AI Model Monitoring is specifically designed to automate this process by tracking prediction drift, feature skew, and accuracy when ground truth labels are available. These features enable scalable and effective continuous monitoring, making options 1 and 4 the correct answers.
- A. Correct.
Correct. Vertex AI Model Monitoring provides automated tracking of prediction drift and feature skew, which are essential for identifying data distribution changes over time.
- B. Incorrect.
Incorrect. While retraining pipelines are important, this approach does not directly contribute to continuous evaluation or monitoring. It addresses model retraining instead of identifying issues like drift or skew.
- C. Incorrect.
Incorrect. Manually analyzing logs is time-consuming and error-prone, making it an inefficient and non-scalable solution for continuous evaluation.
- D. Correct.
Correct. Vertex AI Model Monitoring allows you to use ground truth labels (when available) to calculate metrics like accuracy, which is a key part of continuous evaluation.
- E. Incorrect.
Incorrect. Monitoring latency is important for performance evaluation, but it does not directly address continuous evaluation of the model's prediction quality or data issues like drift or skew.