Databricks Machine Learning Professional Question 264
Select 3You are tasked with monitoring a deployed machine learning model in production to ensure it continues to perform well over time. During your monitoring, you notice that the distribution of input features has shifted significantly compared to the training data. Which of the following solutions would comprehensively address this drift?
- A
Set up automated alerts based on statistical tests for data drift and retrain the model if drift is detected.
- B
Monitor only the model's prediction accuracy and retrain the model when accuracy drops below a predefined threshold.
- C
Implement feature importance monitoring to identify which features contribute most to the drift and adjust the data preprocessing pipeline accordingly.
- D
Perform regular exploratory data analysis (EDA) on production input data to detect changes, without automating drift detection.
- E
Use a baseline model trained on newer production data to compare performance with the deployed model and determine if retraining is necessary.
Show answer and explanation
Correct answers: A, C, E
Explanation
Comprehensive drift solutions should address data drift proactively and systematically. Automating drift detection with statistical tests (Option 1), analyzing feature importance to manage preprocessing pipelines (Option 3), and using baseline models trained on newer data (Option 5) are effective strategies to monitor and respond to drift. Solely relying on accuracy metrics (Option 2) or manual EDA (Option 4) are insufficient for comprehensive drift management in production.
- A. Correct.
This is correct because automating drift detection using statistical tests ensures timely identification of shifts in feature distributions, and retraining the model helps address the drift effectively.
- B. Incorrect.
This is incorrect because solely monitoring model accuracy does not address feature or data drift directly. Drift can occur without an immediate impact on accuracy, leading to delayed detection.
- C. Correct.
This is correct because monitoring feature importance helps pinpoint which features are driving the drift, allowing for targeted adjustments to the data preprocessing pipeline.
- D. Incorrect.
This is incorrect because while exploratory data analysis (EDA) can help detect drift, it is not a comprehensive or automated solution for production environments.
- E. Correct.
This is correct because using a baseline model based on newer production data helps assess the need for retraining and provides a practical way to evaluate drift impact on model performance.