Databricks Machine Learning Professional Question 272
Single answerA retail company has deployed a machine learning model to predict customer purchase likelihoods based on their browsing history. Over time, they notice that the model's predictions are becoming less accurate, and the distribution of customer behaviors in the new data significantly differs from the data the model was originally trained on. When is retraining and deploying an updated model likely the most appropriate solution?
- A
When the model's performance metrics, such as accuracy or F1 score, have dropped below an acceptable threshold due to changes in customer behavior.
- B
When new features are added to the dataset that were not present during the initial training phase.
- C
When the development team identifies that the model's hyperparameters were not optimized during the original training process.
- D
When the training dataset initially used to build the model contained missing or incomplete data.
Show answer and explanation
Correct answer: A
Explanation
Retraining and deploying an updated model is a common solution when drift, such as a shift in the data distribution or changing patterns in customer behavior, causes performance degradation. In this scenario, the drop in model performance and the observed changes in customer behavior indicate concept drift. Retraining the model on the new data distribution can help restore performance, making it the most appropriate solution.
- A. Correct.
Performance degradation caused by changes in customer behavior suggests concept drift, which retraining on updated data can address. This is a clear indication that retraining and redeployment are necessary.
- B. Incorrect.
While adding new features could improve the model, it does not necessarily indicate drift. Retraining due to drift is primarily related to changes in data distribution or relationships, not the addition of new features.
- C. Incorrect.
Hyperparameter optimization issues during the original training process are not related to drift. This is a concern about initial model training quality, not an ongoing degradation in performance.
- D. Incorrect.
Incomplete or missing data in the initial training dataset is an issue of data quality and does not directly relate to the concept of drift or the need for retraining due to changes in data distribution.