Databricks Machine Learning Professional Question 254
Select 2You are monitoring a deployed machine learning model that predicts customer churn. One of the categorical features, 'customer_region', has shown changes in its distribution over time. You want to implement a robust monitoring solution for detecting feature drift in 'customer_region'. Why are statistical tests a better choice than simple summary statistics in this scenario?
- A
Statistical tests can detect subtle changes in the distribution of categorical values, even when summary statistics like mode remain stable.
- B
Statistical tests are computationally faster than calculating summary statistics for categorical features.
- C
Statistical tests provide formal significance levels, helping to quantify the likelihood that detected drift is due to random chance.
- D
Simple summary statistics like mode or frequency count are sufficient for detecting all types of categorical feature drift.
Show answer and explanation
Correct answers: A, C
Explanation
Statistical tests are more robust for detecting categorical feature drift because they analyze the entire distribution of values and provide formal significance metrics, such as p-values, to assess the likelihood of observed changes. Simple summary statistics, like mode or frequency counts, can miss subtle distributional changes, especially in less frequent categories.
- A. Correct.
Correct: Statistical tests, such as chi-square tests, can identify subtle changes in the distribution of categorical features, even if simple statistics like the mode or frequency of the most common value remain unchanged.
- B. Incorrect.
Incorrect: Statistical tests are generally more computationally expensive than calculating simple summary statistics, as they involve comparing full distributions.
- C. Correct.
Correct: Statistical tests provide a formal framework, such as p-values, to quantify the likelihood of observed drift being due to random chance, making them more robust in monitoring scenarios.
- D. Incorrect.
Incorrect: Simple summary statistics may fail to capture nuanced shifts in the distribution, such as changes in less frequent categories, making them insufficient for robust drift detection.