MLS-C01 Question 264
Select 3You are deploying a machine learning model into production using Amazon SageMaker. During inference, you notice that some requests fail intermittently, and you want to monitor these errors to ensure the model's reliability over time. Which combination of steps would you take to build an error monitoring solution for your deployed model?
- A
Enable Amazon CloudWatch Logs for the SageMaker endpoint to capture invocation errors.
- B
Use Amazon SageMaker Model Monitor to detect anomalies in input data and model predictions.
- C
Configure AWS Lambda to automatically retry failed inference requests and log errors to Amazon CloudWatch.
- D
Set up an Amazon SNS topic to send notifications when errors cross a predefined threshold in CloudWatch Metrics.
- E
Enable SageMaker endpoint auto-scaling to handle high traffic and reduce errors.
Show answer and explanation
Correct answers: A, B, D
Explanation
To build an effective error monitoring solution for a SageMaker model, you need to capture and analyze error details (CloudWatch Logs), detect anomalies that might lead to errors (Model Monitor), and implement notification mechanisms for real-time alerts when errors exceed acceptable thresholds (SNS with CloudWatch Metrics). These steps collectively ensure you can detect and respond to errors promptly, maintaining model reliability.
- A. Correct.
Enabling Amazon CloudWatch Logs for the SageMaker endpoint allows you to capture details about invocation errors, which is critical for debugging and monitoring error trends.
- B. Correct.
Amazon SageMaker Model Monitor can detect anomalies in input data and model predictions, which might indicate potential sources of errors during inference.
- C. Incorrect.
While AWS Lambda can retry failed requests and log errors, this option is not directly related to building an error monitoring solution. It focuses more on handling failures rather than monitoring them.
- D. Correct.
Setting up an Amazon SNS topic to send notifications when error rates cross a threshold allows you to stay informed in real-time and take corrective action promptly.
- E. Incorrect.
Enabling SageMaker endpoint auto-scaling helps handle high traffic but does not directly address error monitoring. This option is more related to scaling and availability.