MLA-C01 Question 382
Select 3You are a Machine Learning Engineer responsible for deploying a classification model in production using Amazon SageMaker. To ensure that the model continues to perform well over time, you want to monitor both the data quality and model performance. Which of the following techniques can you implement to achieve this?
- A
Enable Amazon SageMaker Model Monitor to track data drift and anomalies in the input data.
- B
Use Amazon CloudWatch to schedule retraining of the model daily, regardless of data changes.
- C
Set up Amazon SageMaker Model Monitor to evaluate model predictions against a baseline and detect model drift.
- D
Regularly compare the model's prediction accuracy using a labeled dataset from production.
- E
Rely on the training dataset's metrics to evaluate long-term performance in production.
Show answer and explanation
Correct answers: A, C, D
Explanation
Monitoring data quality and model performance in production is a critical aspect of maintaining effective ML systems. Amazon SageMaker Model Monitor is a powerful tool that can detect data and model drift. Additionally, using labeled production data to verify prediction accuracy ensures that the model remains robust under real-world conditions. Relying on static training data or retraining without monitoring does not address these concerns effectively.
- A. Correct.
Correct: Amazon SageMaker Model Monitor is specifically designed to monitor data quality, including detecting data drift and anomalies in input data.
- B. Incorrect.
Incorrect: Scheduling retraining daily without checking for data or model drift can lead to unnecessary compute costs and does not directly monitor data quality or model performance.
- C. Correct.
Correct: Amazon SageMaker Model Monitor can also monitor model quality by comparing predictions against a baseline to identify model drift.
- D. Correct.
Correct: Comparing the model's predictions using a labeled dataset from production ensures that the model is evaluated against real-world data, allowing for insights into its performance.
- E. Incorrect.
Incorrect: Relying solely on the training dataset's metrics is not sufficient for long-term monitoring because production data can differ significantly from training data, leading to potential performance degradation.