Databricks Machine Learning Professional Question 248
Select 2You are tasked with monitoring feature drift in a machine learning model used for predicting housing prices. The data has a numeric feature, 'Median_Income', that is critical to the model's performance. Why might statistical tests (e.g., Kolmogorov-Smirnov test) be more robust than simple summary statistics (e.g., mean, variance) for detecting drift in 'Median_Income'?
- A
Statistical tests can identify distributional changes that summary statistics might miss.
- B
Statistical tests are specifically designed to detect outliers in the dataset.
- C
Summary statistics are only effective when the feature distributions are normal.
- D
Statistical tests can compare the entire distribution of the feature between two datasets.
Show answer and explanation
Correct answers: A, D
Explanation
Statistical tests are more robust for detecting numeric feature drift because they analyze the entire distribution of a feature, capturing subtle changes that simple summary statistics like mean and variance might miss. While summary statistics are easy to compute, they may fail to detect shifts in other aspects of the distribution, such as skewness or multimodality, which can significantly affect model performance.
- A. Correct.
Correct: Statistical tests examine the entire distribution of a feature, making them sensitive to subtle changes in the shape of the distribution that summary statistics might not capture.
- B. Incorrect.
Incorrect: While statistical tests may detect distributional changes caused by outliers, they are not specifically designed for outlier detection.
- C. Incorrect.
Incorrect: Summary statistics can be used with non-normal distributions as well, but they provide limited information about the overall distributional changes.
- D. Correct.
Correct: Statistical tests, such as the Kolmogorov-Smirnov test, are designed to compare entire distributions, making them more robust for detecting feature drift.