Google Professional Machine Learning Engineer Question 505
Select 3Google Cloud PlatformYou are deploying a machine learning model for predicting customer churn in a subscription service. The model uses features such as user activity, subscription type, and support ticket history. Over time, you notice a drop in model performance during live predictions. You suspect feature attribution drift might be a contributing factor. Which of the following steps should you take to monitor and address feature attribution drift?
- A
Enable Vertex AI Model Monitoring to track feature attribution drift metrics.
- B
Compare the feature importance of the training data with the live inference data using SHAP or similar tools.
- C
Retrain the model immediately with the latest live data to mitigate the effects of drift.
- D
Set up alerts for significant deviations in the distribution of feature attribution scores between training and live data.
- E
Ignore feature attribution drift and focus solely on monitoring data drift in the input features.
Show answer and explanation
Correct answers: A, B, D
Explanation
Feature attribution drift occurs when the importance of features in making predictions changes over time, potentially leading to degraded model performance. Addressing this requires monitoring tools like Vertex AI Model Monitoring, analyzing feature importance using techniques such as SHAP, and setting up alerts for observed deviations. These steps help detect and mitigate drift effectively before retraining the model or implementing other corrective measures.
- A. Correct.
Enabling Vertex AI Model Monitoring provides automated tools to track and alert on various types of drift, including feature attribution drift, which makes it a critical step.
- B. Correct.
Comparing feature importance (e.g., using SHAP values) between training and live inference data helps to identify shifts in how features contribute to model predictions, which is essential for monitoring feature attribution drift.
- C. Incorrect.
Retraining the model immediately is not a recommended first step without proper analysis, as it may lead to overfitting or overlooking the root cause of the drift.
- D. Correct.
Setting up alerts for deviations in feature attribution scores helps in proactively identifying and addressing drift issues, making it an important step in the monitoring process.
- E. Incorrect.
Ignoring feature attribution drift is not advisable, as it can impact model interpretability and performance. Monitoring data drift in input features alone is not sufficient to address attribution drift.