Databricks Machine Learning Professional Question 273
Single answerYou are monitoring a deployed machine learning model that predicts customer churn for a subscription-based service. Over time, you observe that the model's accuracy on recent data has significantly decreased compared to its accuracy on the validation set during training. Upon further analysis, you discover that the distribution of customer behavior features has shifted due to a major change in the company's pricing strategy. What is the most appropriate course of action?
- A
Retrain the model using the most recent data and redeploy the updated model.
- B
Increase the size of the original training dataset by including synthetic examples.
- C
Adjust the model's hyperparameters to better fit the new data distribution.
- D
Deploy an ensemble model that combines predictions from the old and new data distributions.
Show answer and explanation
Correct answer: A
Explanation
Data drift occurs when the distribution of input features or target variables changes over time, leading to decreased model performance. In this scenario, the change in customer behavior due to the new pricing strategy has caused data drift. Retraining the model with the most recent data ensures that the model learns from the updated data distribution and can make accurate predictions under the new conditions.
- A. Correct.
Retraining the model with the most recent data addresses the issue of data drift caused by the change in customer behavior. This is the most appropriate solution to adapt the model to the new data distribution.
- B. Incorrect.
Adding synthetic examples to the original training dataset does not directly address the issue of distribution drift caused by the pricing strategy change. It may not capture the true patterns in the recent data.
- C. Incorrect.
Adjusting hyperparameters cannot resolve the underlying issue of data drift. Hyperparameter tuning is typically used to improve performance on the same data distribution, not on a shifted one.
- D. Incorrect.
Deploying an ensemble model might help in some cases, but it does not directly address the root cause of the drift. Additionally, if the old data distribution is no longer relevant, its inclusion could reduce the model's performance.