Databricks Machine Learning Professional Question 274
Select 2You are monitoring a deployed machine learning model used to predict product demand. Recently, you have observed a significant drop in the model's prediction accuracy on the latest production data. Further analysis shows that customer purchasing patterns have shifted due to a recent market trend. Which of the following scenarios would indicate that retraining and deploying an updated model is a probable solution to address this issue?
- A
The distribution of input features in the production data no longer matches the training data.
- B
The model's hyperparameters were not optimally tuned during the initial training process.
- C
A new dataset has become available that reflects the current purchasing patterns of customers.
- D
The current model shows consistently high accuracy on the training dataset but poor performance on production data.
- E
The model's prediction latency has increased significantly due to increased production data volume.
Show answer and explanation
Correct answers: A, C
Explanation
Retraining and deploying an updated model is a probable solution to drift when there is evidence of a change in data distribution (data drift) or when new data reflecting current trends is available for training. These scenarios indicate that the model needs to adapt to changes in the environment it operates in.
- A. Correct.
When the distribution of input features in production data changes compared to the training data, it indicates a data drift. Retraining the model with updated data can address this issue.
- B. Incorrect.
Suboptimal hyperparameter tuning during initial training is unrelated to changes in production data. This would require model tuning, not necessarily retraining due to drift.
- C. Correct.
Availability of a new dataset reflecting current customer patterns suggests that retraining the model with this updated data could improve its performance and adapt to the new trends.
- D. Incorrect.
High accuracy on the training dataset but poor performance on production data suggests overfitting, not drift. This requires addressing overfitting rather than retraining for drift.
- E. Incorrect.
An increase in prediction latency is unrelated to data drift or model accuracy. This is typically an infrastructure or performance optimization issue.