Databricks Machine Learning Professional Question 245
Select 3You are monitoring a machine learning model deployed in production to predict customer preferences based on categorical features such as 'location' and 'membership type.' Over time, you observe a significant distribution shift in the 'membership type' feature. Which of the following approaches can help you identify and address this categorical feature drift using simple solutions?
- A
Analyze the mode of the 'membership type' feature over time to identify changes in the most frequently occurring category.
- B
Check the unique values of the 'membership type' feature in the incoming data and compare them to the training data.
- C
Use advanced statistical tests like the Kolmogorov-Smirnov test to quantify the drift in 'membership type.'
- D
Monitor the percentage of missing values in the 'membership type' feature to detect any significant changes.
- E
Re-train the model immediately whenever a new unique value appears in the 'membership type' feature.
Show answer and explanation
Correct answers: A, B, D
Explanation
Feature drift in categorical variables can often be addressed using simple solutions such as tracking changes in the mode, unique values, and missing values. These approaches allow for the identification of significant shifts without requiring complex statistical methods, making them accessible for routine monitoring and debugging in production environments.
- A. Correct.
Analyzing the mode over time is an effective way to detect changes in the most common category, which can indicate feature drift in categorical data.
- B. Correct.
Checking the unique values helps identify whether new categories have appeared or if expected categories have disappeared, providing direct evidence of drift.
- C. Incorrect.
While advanced statistical tests can detect distribution shifts, they are not categorized as 'simple solutions' and therefore fall outside the scope of this question.
- D. Correct.
Monitoring the percentage of missing values is a straightforward method for identifying anomalies or data quality issues that contribute to feature drift.
- E. Incorrect.
Re-training the model immediately for every new unique value is not practical or necessary. Instead, the situation should first be analyzed to determine the impact of the new value.