MLS-C01 Question 262
Select 3You are a Machine Learning Engineer managing an ML model deployed using Amazon SageMaker. You notice that the model's performance has degraded in production due to data drift and inference errors. You want to implement a solution to monitor these issues and receive alerts for any anomalies. Which of the following steps will help you achieve this?
- A
Enable Amazon SageMaker Model Monitor to track data drift, feature distributions, and quality metrics.
- B
Use Amazon CloudWatch to create alarms based on metrics generated by SageMaker Model Monitor.
- C
Deploy an AWS Lambda function to retrain the model automatically whenever an anomaly is detected.
- D
Use Amazon Simple Notification Service (SNS) to send alerts when anomalies are detected by SageMaker Model Monitor.
- E
Configure Amazon Macie to monitor sensitive data in the input data streams for the model.
Show answer and explanation
Correct answers: A, B, D
Explanation
To build an error monitoring solution for an ML model deployed with Amazon SageMaker, you can use SageMaker Model Monitor to track data drift, feature distributions, and quality metrics. Metrics from Model Monitor can be sent to Amazon CloudWatch, where you can create alarms. To receive notifications about anomalies, you can use Amazon SNS. These services together enable a comprehensive monitoring and alerting system for ML models in production.
- A. Correct.
Correct. Amazon SageMaker Model Monitor can be used to track data drift, feature distributions, and model quality metrics, which is essential for identifying potential issues in production.
- B. Correct.
Correct. Amazon CloudWatch can be used to create alarms based on the metrics emitted by SageMaker Model Monitor, enabling you to monitor for anomalies and take action.
- C. Incorrect.
Incorrect. While retraining the model might be a long-term solution, automatically triggering retraining with AWS Lambda is not recommended without proper validation. This step is not directly related to error monitoring.
- D. Correct.
Correct. Amazon SNS can send notifications or alerts when anomalies are detected by SageMaker Model Monitor, informing the team of potential issues.
- E. Incorrect.
Incorrect. Amazon Macie is used for data security and compliance monitoring, not for tracking model performance or input data drift.