Databricks Machine Learning Professional Question 241
Select 2A machine learning team is monitoring a production model that predicts house prices based on features such as square footage, number of rooms, and lot size. Recently, they suspect that the 'square footage' feature may be experiencing drift due to changes in the data distribution. The team wants to implement summary statistic monitoring as a simple solution to detect numeric feature drift. Which of the following steps should they take?
- A
Monitor the mean and standard deviation of the 'square footage' feature over time and compare them against baseline values.
- B
Track the count of missing values in the 'square footage' feature and compare it to the training dataset.
- C
Use the Kolmogorov-Smirnov (K-S) test to compare the current 'square footage' distribution to the training distribution.
- D
Visualize the distribution of 'square footage' in production using histograms and perform qualitative analysis.
- E
Set up threshold-based alerts to trigger notifications if summary statistics of 'square footage' deviate significantly from baseline.
Show answer and explanation
Correct answers: A, E
Explanation
Summary statistic monitoring involves tracking simple metrics like mean, standard deviation, or other basic statistics of a numeric feature and comparing them to baseline values from the training data. Alerts can be configured to notify the team when these metrics deviate significantly, making it an effective and straightforward solution for detecting numeric feature drift.
- A. Correct.
Monitoring the mean and standard deviation over time is a key part of summary statistic monitoring for numeric feature drift. These statistics provide a simple yet effective way to detect changes in the feature's distribution.
- B. Incorrect.
Tracking missing values is important for data quality monitoring but does not directly address numeric feature drift.
- C. Incorrect.
Using the Kolmogorov-Smirnov test is a statistical approach for detecting distributional drift, but it is not considered a simple summary statistic monitoring method.
- D. Incorrect.
Visualizing distributions can help in understanding the data but does not constitute an automated or programmatic summary statistic monitoring approach.
- E. Correct.
Setting up threshold-based alerts for deviations in summary statistics (e.g., mean and standard deviation) helps automate drift detection and aligns with the goal of summary statistic monitoring.