Databricks Machine Learning Professional Question 219
Select 3You have deployed a machine learning model to production using Databricks. After several weeks in production, the model's predictions start to show performance degradation. Upon investigation, you notice that the underlying data distribution has changed over time. Which of the following steps should you take to monitor and address such issues effectively?
- A
Implement data drift detection to monitor changes in the input data distribution.
- B
Use a model registry to track and version new model updates.
- C
Set up alerting mechanisms to notify stakeholders when prediction performance drops below a threshold.
- D
Retrain the model on the new data without monitoring its impact on prediction accuracy.
- E
Log both input features and model predictions for continuous monitoring and debugging.
Show answer and explanation
Correct answers: A, C, E
Explanation
To address performance degradation due to changes in data distribution, it is critical to implement data drift detection, monitor model performance through logging, and establish alerting mechanisms for timely intervention. These steps ensure that performance issues are identified and resolved promptly, maintaining the reliability of the production model.
- A. Correct.
Correct: Data drift detection is essential to monitor changes in the input data distribution, which could lead to model performance degradation over time.
- B. Incorrect.
Partially correct but not sufficient on its own: While using a model registry helps manage model versions, it does not directly address monitoring or diagnosing performance degradation due to data drift.
- C. Correct.
Correct: Setting up alerting mechanisms ensures that stakeholders are notified when the model underperforms, enabling timely intervention.
- D. Incorrect.
Incorrect: Retraining the model on new data without monitoring its impact on accuracy could lead to unintended consequences, such as overfitting or poor generalization.
- E. Correct.
Correct: Logging input features and predictions supports continuous monitoring, enabling the detection of issues like data drift or prediction errors.