Databricks Machine Learning Professional Question 226
Single answerYou are monitoring the performance of a deployed machine learning model that predicts customer churn. Over time, you observe that predictions are less accurate. Upon investigation, you identify that the distribution of customer age, which is one of the input features, has shifted significantly compared to the training data. However, the distribution of the churn labels in the evaluation dataset has remained consistent. Which of the following statements correctly explains the situation?
- A
This is an example of label drift, as the model's accuracy is decreasing.
- B
This is an example of feature drift, as the distribution of an input feature has shifted.
- C
This is an example of label drift, as the distribution of labels has remained consistent.
- D
This is an example of feature drift, as the distribution of churn label predictions has shifted.
Show answer and explanation
Correct answer: B
Explanation
This scenario highlights feature drift, where the distribution of an input feature (customer age) has shifted compared to the training data. Feature drift can negatively impact model performance because the model is trained on a specific feature distribution and may struggle when that distribution changes. Label drift, on the other hand, refers to shifts in the target label distribution, which is not the case here.
- A. Incorrect.
Incorrect. Label drift refers to changes in the distribution of the target labels over time, which is not the case here. The model's accuracy is decreasing due to a shift in the input feature distribution, not the labels.
- B. Correct.
Correct. Feature drift occurs when the distribution of input features shifts over time. In this case, the distribution of customer age, an input feature, has shifted, causing a decrease in model performance.
- C. Incorrect.
Incorrect. Label drift is characterized by a change in the distribution of the target labels, which is not observed here. The target labels have remained consistent in distribution.
- D. Incorrect.
Incorrect. Feature drift refers to changes in the input feature distribution, not the distribution of the model's predictions. The observed issue is related to the input feature distribution, not the predicted labels.