Google Professional Machine Learning Engineer Question 498
Select 3Google Cloud PlatformYou are deploying a machine learning model to production on Google Cloud. After deployment, you notice a significant drop in model performance compared to what was observed during training. You suspect training-serving skew. Which of the following steps should you take to monitor and address this issue?
- A
Enable Vertex AI's feature monitoring to compare feature statistics between training and serving data.
- B
Log predictions and actual labels in production and compute the model's performance metrics over time.
- C
Implement an automated pipeline to retrain the model every time new data is collected in production.
- D
Use Dataflow to preprocess the training and serving data separately to account for differences.
- E
Validate that the feature engineering transformations applied during training are consistent with those applied during serving.
Show answer and explanation
Correct answers: A, B, E
Explanation
Training-serving skew occurs when there are discrepancies between the data used for training the model and the data used during serving. This can lead to degraded model performance. To monitor and address this issue, it is essential to use tools like Vertex AI's feature monitoring to compare feature distributions, validate the consistency of feature engineering transformations, and log production metrics to track performance. Automated retraining and separate preprocessing pipelines don't directly address training-serving skew and can even introduce additional issues.
- A. Correct.
Correct. Vertex AI's feature monitoring can help detect differences in feature distributions between training and serving data, which is a common cause of training-serving skew.
- B. Correct.
Correct. Logging predictions and actual labels allows you to monitor the model's performance in production and identify potential issues caused by training-serving skew.
- C. Incorrect.
Incorrect. While retraining the model is useful for handling data drift, it does not directly address or monitor training-serving skew.
- D. Incorrect.
Incorrect. Preprocessing training and serving data separately can introduce inconsistencies, which may exacerbate training-serving skew instead of solving it.
- E. Correct.
Correct. Ensuring consistency in feature engineering transformations between training and serving is critical to preventing training-serving skew.