Databricks Machine Learning Professional Question 224
Select 3A machine learning team deployed a model to predict customer churn in a subscription service. After monitoring the model for several months, they notice a drop in model performance. Upon investigation, they discover that the distribution of the 'subscription duration' feature has shifted significantly, while the distribution of the 'churn' label has remained stable. Which of the following statements correctly identify the observed issue and distinguish it from other types of drift?
- A
This is an example of feature drift because the distribution of an input feature has changed.
- B
This is an example of label drift because the distribution of the target variable has remained constant.
- C
Feature drift affects the input features and can degrade model performance even if label distribution remains stable.
- D
Label drift occurs when the distribution of the target variable changes, potentially impacting model performance.
- E
Feature drift and label drift are synonymous terms that describe the same phenomenon.
Show answer and explanation
Correct answers: A, C, D
Explanation
The scenario describes feature drift because the distribution of the 'subscription duration' input feature has shifted while the label distribution remains stable. Feature drift impacts the input features, potentially degrading model performance, whereas label drift relates to changes in the distribution of the target variable. Understanding the distinction between these two types of drift is critical for diagnosing and addressing model performance issues in production.
- A. Correct.
Correct: Feature drift refers to changes in the distribution of input features over time, which is the case here since the 'subscription duration' feature distribution has shifted.
- B. Incorrect.
Incorrect: Label drift refers to changes in the distribution of the target variable (e.g., 'churn'), but in this case, the label distribution has not changed.
- C. Correct.
Correct: Feature drift can cause model performance to degrade because the model may no longer generalize well to the new input feature distribution.
- D. Correct.
Correct: Label drift is explicitly defined as a change in the distribution of the target variable, which is not observed in this scenario but is an important concept to distinguish.
- E. Incorrect.
Incorrect: Feature drift and label drift are distinct concepts, not synonymous. This statement is inaccurate.