Databricks Machine Learning Professional Question 232
Single answerA retail company has deployed a machine learning model to predict customer churn. Over time, the company observes that the model's accuracy on recent data has significantly decreased compared to when it was initially deployed. Upon investigation, they find that customers' purchasing behavior has changed due to a new competitor entering the market. What is the most likely cause of the model's degraded performance?
- A
The model has overfitted to the training data and cannot generalize to new samples.
- B
Concept drift has occurred, where the statistical properties of the target variable or features have changed over time.
- C
The data pipeline feeding the model has broken, causing incomplete or incorrect data to be used for predictions.
- D
The hyperparameters of the machine learning model need to be re-tuned to adapt to the new data.
Show answer and explanation
Correct answer: B
Explanation
Concept drift occurs when the statistical properties of the input features or target variable change over time, causing the model to perform poorly on new data. In this scenario, the introduction of a new competitor altered customer purchasing behavior, leading to a shift in data distribution that the model was not trained to handle. Recognizing concept drift is crucial for maintaining model efficacy in dynamic environments.
- A. Incorrect.
Overfitting occurs when a model performs well on training data but poorly on unseen data. However, this scenario describes a time-related change in data distribution, not overfitting.
- B. Correct.
Concept drift refers to changes in the statistical properties of features or the target variable over time, leading to a mismatch between the training data and real-world conditions. This is the most likely explanation given the change in customer behavior.
- C. Incorrect.
A broken data pipeline could also result in degraded model performance, but the scenario specifies changes in customer behavior, not issues with the data pipeline.
- D. Incorrect.
Hyperparameter tuning optimizes model performance but does not address the underlying issue of concept drift caused by changes in data distribution.