Google Professional Machine Learning Engineer Question 500
Select 3Google Cloud PlatformYou are working on a machine learning model hosted on Google Cloud AI Platform. After deploying the model to production, you notice that the model's predictions seem to differ significantly from what was observed during training. You suspect a training-serving skew. Which of the following actions should you take to detect and monitor training-serving skew effectively?
- A
Enable Vertex AI's feature monitoring to track feature statistics in production and compare them with training data.
- B
Log predictions and labels in production, and compare metrics with those from training data.
- C
Regularly retrain the model using production data to ensure consistency.
- D
Use Dataflow to preprocess both training and serving data with the same transformations.
- E
Set up a CI/CD pipeline to automatically retrain the model when data drift is detected.
Show answer and explanation
Correct answers: A, B, D
Explanation
Training-serving skew occurs when there are differences between the data or transformations used during training and those in production, leading to inconsistent predictions. Effective monitoring strategies include comparing feature distributions (using tools like Vertex AI feature monitoring) and logging/monitoring model predictions and metrics in production. Ensuring consistent preprocessing between training and serving (e.g., with Dataflow) is another vital step to minimize potential skew.
- A. Correct.
Enabling Vertex AI's feature monitoring allows you to track feature distributions in production and compare them with the training data, helping to identify differences that could cause skew.
- B. Correct.
Logging predictions and labels in production and comparing key metrics with training metrics helps identify performance degradation caused by skew.
- C. Incorrect.
Regularly retraining the model is a good practice for handling concept drift but does not directly address detecting or monitoring training-serving skew.
- D. Correct.
Using Dataflow to preprocess both training and serving data with identical transformations ensures consistency and reduces the likelihood of skew caused by differences in preprocessing.
- E. Incorrect.
Setting up a CI/CD pipeline for automatic retraining is a good MLOps practice for managing model lifecycle but is not a method for detecting or monitoring training-serving skew.