MLS-C01 Question 386
Select 2You have deployed a machine learning model in production using Amazon SageMaker. Over time, you notice that the model's predictions are becoming less accurate due to changes in the underlying data distribution. What steps can you take to monitor and address this issue effectively?
- A
Enable Amazon SageMaker Model Monitor to detect data drift and quality issues in the input data.
- B
Manually log prediction errors and compare them against historical performance metrics.
- C
Use Amazon CloudWatch to monitor model latency and throughput metrics.
- D
Set up a baseline using historical training data with Amazon SageMaker Model Monitor and schedule periodic monitoring jobs.
- E
Enable automatic retraining of the model using Amazon SageMaker Pipelines whenever a threshold of prediction errors is exceeded.
Show answer and explanation
Correct answers: A, D
Explanation
Monitoring model performance in production is crucial for ensuring reliable predictions over time. Amazon SageMaker Model Monitor is a key tool provided by AWS to detect and address issues like data drift and data quality degradation automatically. Setting up a baseline and scheduling monitoring jobs ensures that deviations in the input data distribution are identified early. While other options like logging errors or monitoring operational metrics can provide insights, they don't directly address data drift or ensure automated monitoring.
- A. Correct.
Amazon SageMaker Model Monitor is specifically designed to detect issues like data drift, which can cause model performance degradation.
- B. Incorrect.
Manually logging prediction errors can be useful but is not an automated or comprehensive solution for monitoring data drift or quality over time.
- C. Incorrect.
Amazon CloudWatch helps track operational metrics like latency and throughput but does not directly address data drift or performance monitoring for a model.
- D. Correct.
Setting up a baseline with Amazon SageMaker Model Monitor allows you to compare current data to historical patterns and detect deviations automatically, which is critical for monitoring model performance.
- E. Incorrect.
While automatic retraining can be useful, it requires additional configuration and is not an inherent feature of Amazon SageMaker Model Monitor. It is not directly related to monitoring but rather to mitigation of issues.