Databricks Machine Learning Professional Question 252
Select 2You are tasked with monitoring feature drift in a machine learning model deployed in production. One of the features, 'customer_region', is categorical and has shown significant changes in its distribution over time. Why would hypothesis tests be a more robust solution for detecting drift in this feature compared to simple summary statistics such as mode or unique value counts?
- A
Hypothesis tests can statistically quantify whether observed changes in the categorical distribution are significant.
- B
Summary statistics like mode or unique value counts are sufficient to identify complex changes in categorical distributions.
- C
Hypothesis tests are specifically designed to detect shifts, even when the changes are subtle and not observable through basic summaries.
- D
Simple summary statistics are computationally expensive compared to hypothesis tests for categorical data.
Show answer and explanation
Correct answers: A, C
Explanation
Hypothesis tests, such as the Chi-Square test, provide a robust mechanism to statistically test for significant changes in the distribution of categorical features. This approach is more reliable than summary statistics, which can only provide basic insights and may fail to capture subtle or complex changes in distributions. For robust monitoring in production systems, where undetected drift can degrade model performance, hypothesis testing is a more suitable option.
- A. Correct.
Correct: Hypothesis tests, such as the Chi-Square test, provide a statistical measure to determine if the observed changes in the categorical distribution are significant, which is critical for robust monitoring.
- B. Incorrect.
Incorrect: Summary statistics like mode or unique value counts are limited in their ability to capture complex or subtle changes in categorical distributions and are generally not sufficient for robust drift detection.
- C. Correct.
Correct: Hypothesis tests are designed to detect even subtle shifts in the underlying distribution, which may not be evident through simple metrics like mode or unique counts.
- D. Incorrect.
Incorrect: Summary statistics are generally computationally cheaper than hypothesis tests, as they involve simple counting operations rather than statistical inference.