Databricks Machine Learning Professional Question 233
Single answerA retail company has deployed a machine learning model to predict customer purchase likelihood based on historical data. Over time, the company notices a significant drop in the model's accuracy, coinciding with a change in customer behavior due to the introduction of a new product line. Which of the following best describes the issue and its potential impact on the model?
- A
The model is experiencing overfitting, as it memorized patterns in the training data that do not generalize to new customer behavior.
- B
The model is affected by concept drift, as the relationship between the input features and the target variable has changed over time.
- C
The model has data leakage, as it likely used information unavailable during prediction in the training dataset.
- D
The model's hyperparameters need to be tuned to adapt to new customer behavior.
Show answer and explanation
Correct answer: B
Explanation
The issue described is a classic example of concept drift. Customer behavior changes due to the introduction of a new product line, altering the relationship between input features and the target variable. This causes the model to underperform since it was trained on historical data that no longer reflects the current patterns. Identifying and addressing concept drift is critical to maintaining model efficacy over time.
- A. Incorrect.
Overfitting refers to a model performing well on training data but poorly on unseen or test data due to memorizing noise or irrelevant details. This does not explain why the model's accuracy dropped over time as customer behavior changed.
- B. Correct.
Concept drift occurs when the statistical properties of the target variable or the relationship between input features and target variable change over time, which fits the described scenario. This change impacts the model's performance.
- C. Incorrect.
Data leakage occurs when information from outside the training dataset that would not be available during prediction is used in training. This is unrelated to the issue described, which is a temporal change in customer behavior.
- D. Incorrect.
Hyperparameter tuning optimizes a model's performance but does not address the underlying issue of a shift in the data distribution or feature-target relationship.