Databricks Machine Learning Professional Question 216
Select 3You have deployed a machine learning model to predict customer churn. After deployment, you observe a significant drop in the model's accuracy. Which of the following steps should you take to monitor and address the issue?
- A
Implement data drift monitoring to detect changes in input data distribution.
- B
Retrain the model without checking for changes in the underlying data distributions.
- C
Set up monitoring for model performance metrics, such as precision and recall, on production data.
- D
Use feature importance analysis to identify features that might no longer be predictive.
- E
Ignore the accuracy drop and wait for more data to validate the issue.
Show answer and explanation
Correct answers: A, C, D
Explanation
When a deployed model experiences a drop in accuracy, it is important to monitor for data drift, evaluate production metrics, and analyze feature importance to diagnose and address the issue effectively. These actions help ensure the model remains reliable and performant in production. Ignoring the issue or retraining without proper analysis can lead to suboptimal outcomes.
- A. Correct.
Implementing data drift monitoring is critical to detect if the input data distribution has changed compared to the training data, which can lead to performance degradation.
- B. Incorrect.
Retraining the model without understanding the root cause of the issue (e.g., data drift or concept drift) is not advisable and may not address the issue effectively.
- C. Correct.
Monitoring metrics like precision and recall on production data helps identify and quantify model performance changes over time.
- D. Correct.
Feature importance analysis can help identify which features are no longer predictive and may be causing the drop in model performance.
- E. Incorrect.
Ignoring the issue and waiting for more data is not proactive and could lead to prolonged poor performance, negatively impacting the use case.