Databricks Machine Learning Professional Question 276
Single answerYou recently updated a machine learning model to improve its performance. To test whether the updated model performs better on more recent data, you have access to the following datasets: the original training data, a validation dataset, and a new dataset containing recent data. What is the most appropriate approach to evaluate the updated model's performance?
- A
Evaluate the updated model using the original training data and compare its metrics against the old model.
- B
Evaluate the updated model using the validation dataset and compare its metrics against the old model.
- C
Evaluate the updated model using the new recent data and compare its metrics against the old model.
- D
Combine the validation dataset and recent data into one dataset, evaluate the updated model on this combined dataset, and compare its metrics to the old model.
Show answer and explanation
Correct answer: C
Explanation
To determine whether the updated model performs better on more recent data, it is critical to evaluate it directly on the recent dataset. This dataset represents the updated conditions and trends the model is intended to handle. Comparing the model's performance on recent data against the old model ensures the evaluation aligns with the goal of testing its improvement in handling current data distributions.
- A. Incorrect.
Using the original training data to evaluate the updated model does not reflect its performance on more recent data, as the training data represents historical trends and may not account for changes in the data distribution.
- B. Incorrect.
Using the validation dataset for evaluation is common during model development, but it may not represent the recent data distribution. This makes it unsuitable for testing performance on more recent data.
- C. Correct.
Evaluating the updated model on the new recent data is the best approach, as it directly tests whether the model performs better on the most relevant and recent data. This ensures the evaluation aligns with the goal of assessing performance on recent trends.
- D. Incorrect.
Combining the validation dataset and recent data into one dataset could dilute the recent data's influence, making it difficult to assess the model's specific performance on the most recent data. This approach is not as targeted as evaluating on the recent data alone.