Databricks Machine Learning Professional Question 229
Select 3A retail company has deployed a machine learning model to predict the demand for products in its stores. Over time, they notice that the model's performance has degraded. Which of the following scenarios could likely explain the observed issue due to feature drift and/or label drift?
- A
The company introduces new product categories that were not present during model training.
- B
The definition of 'high demand' in the training labels is updated to reflect a lower threshold of sales.
- C
The model's hyperparameters were manually updated after deployment without retraining.
- D
Customer purchasing preferences have shifted significantly due to seasonal trends.
- E
The training data used for the model included a bias towards urban stores, but the model is now being used in rural stores.
Show answer and explanation
Correct answers: A, B, D
Explanation
Feature drift and label drift are common causes of model performance degradation in real-world applications. Feature drift occurs when the distribution of input features changes over time (e.g., new product categories or shifts in customer preferences), while label drift occurs when the definition or distribution of the target variable changes (e.g., redefining 'high demand'). Identifying these scenarios helps ensure models remain robust in dynamic environments.
- A. Correct.
Feature drift occurs when new product categories are introduced, as the input features related to products may change significantly compared to the training data.
- B. Correct.
Label drift occurs when the definition or distribution of the target variable (e.g., 'high demand') changes, leading to a mismatch between the training labels and the real-world labels.
- C. Incorrect.
While updating hyperparameters may impact model performance, it is not related to feature or label drift since it does not involve changes to the input features or label definitions.
- D. Correct.
Feature drift can occur if customer preferences shift over time, causing the distribution of input features (e.g., product sales patterns) to differ from the original training data.
- E. Incorrect.
Although this scenario highlights a potential data bias issue, it is not directly caused by feature drift or label drift but rather by a mismatch between the training and operational contexts.