MLA-C01 Question 364
Select 3You are managing a machine learning model deployed on AWS SageMaker for real-time predictions. The model is used for customer churn prediction, and its performance is critical to business decisions. Recently, the model's predictions seem unreliable due to potential data distribution drift. Which approaches can you take to monitor the model inference and detect issues?
- A
Enable Amazon SageMaker Model Monitor to capture and analyze input data for drift over time.
- B
Inspect the endpoint logs in Amazon CloudWatch to manually compare predictions with historical data.
- C
Set up a SageMaker endpoint alarm using Amazon CloudWatch to monitor latency and error rates.
- D
Use Amazon SageMaker Clarify to monitor bias in the model's predictions over time.
- E
Configure a custom Lambda function to preprocess incoming requests and log them for manual inspection.
Show answer and explanation
Correct answers: A, C, D
Explanation
Monitoring model inference requires tools that can automatically identify issues like data drift, bias, and endpoint performance. Amazon SageMaker Model Monitor is ideal for detecting data drift, while SageMaker Clarify helps track prediction bias. Additionally, endpoint metrics like latency and error rates can indicate inference issues, which can be monitored through CloudWatch alarms. These approaches provide a comprehensive strategy for maintaining model reliability and performance.
- A. Correct.
Amazon SageMaker Model Monitor is specifically designed to track data quality and drift in input data, making it one of the best tools for identifying issues with model inference.
- B. Incorrect.
Manually inspecting logs in Amazon CloudWatch can be time-consuming and prone to errors, making it an inefficient approach for monitoring model inference.
- C. Correct.
Setting up a SageMaker endpoint alarm helps monitor endpoint performance metrics like latency and error rates, which could indicate potential issues with the model inference.
- D. Correct.
Amazon SageMaker Clarify is a powerful tool for monitoring bias in predictions, which is an essential aspect of ensuring reliable model inference.
- E. Incorrect.
While a custom Lambda function could log requests, it lacks the automated and scalable capabilities of SageMaker tools for monitoring inference.