Databricks Machine Learning Professional Question 253
Select 2You are tasked with monitoring a machine learning model in production. The model uses a categorical feature for prediction. During a recent review, you noticed that the distribution of this feature in incoming data has shifted significantly compared to the training data. Why might statistical tests, such as Chi-Square tests, provide a more robust solution for monitoring this drift compared to simple summary statistics?
- A
Statistical tests can account for the relationship between categories and their expected frequencies.
- B
Simple summary statistics are only effective for numerical features and cannot be applied to categorical features.
- C
Statistical tests provide a formal significance level to determine whether observed drift is due to random chance.
- D
Simple summary statistics, such as mode or frequency counts, are sufficient to detect feature drift in all scenarios.
Show answer and explanation
Correct answers: A, C
Explanation
Statistical tests, such as the Chi-Square test, are more robust for monitoring categorical feature drift because they evaluate the significance of observed vs. expected changes in the distribution. Simple summary statistics, while useful, lack the formal rigor to determine whether an observed drift is statistically significant. This makes statistical tests better suited for robust monitoring solutions in production environments.
- A. Correct.
Correct. Statistical tests, such as the Chi-Square test, evaluate the distribution of observed vs. expected frequencies for categories, offering deeper insights into drift patterns.
- B. Incorrect.
Incorrect. Simple summary statistics can still be applied to categorical features, such as calculating frequency counts, but they may not fully capture the significance of the drift.
- C. Correct.
Correct. Statistical tests provide a rigorous framework with formal thresholds for detecting whether an observed drift is significant or simply due to random variation.
- D. Incorrect.
Incorrect. While simple summary statistics can reveal basic patterns, they may miss subtle or statistically significant changes in the categorical feature distribution.