MLA-C01 Question 367
Select 3You are managing a live machine learning model deployed on Amazon SageMaker that predicts loan approvals. Recently, you've noticed a significant drop in the accuracy of predictions. To monitor and analyze model inference issues, which of the following actions should you take?
- A
Enable Amazon SageMaker Model Monitor to track data drift and set baseline constraints.
- B
Use Amazon CloudWatch Logs to monitor the latency of the model endpoint.
- C
Implement Amazon SageMaker Clarify to check for bias in inference requests.
- D
Collect inference requests and actual outcomes over time to calculate model performance metrics.
- E
Use AWS CloudTrail to analyze API calls made to the model endpoint.
Show answer and explanation
Correct answers: A, C, D
Explanation
When faced with a drop in model inference accuracy, it is essential to monitor and analyze key aspects such as data drift, bias, and model performance metrics. Amazon SageMaker Model Monitor and Clarify are purpose-built tools for tracking data drift and detecting bias, respectively. Additionally, collecting inference requests and actual outcomes enables you to compute performance metrics to diagnose and address accuracy issues effectively.
- A. Correct.
Correct: Amazon SageMaker Model Monitor can track data drift and help identify if the input data distribution has changed from the original training data, which could cause the accuracy drop.
- B. Incorrect.
Incorrect: While CloudWatch Logs can help monitor latency, it does not directly address inference accuracy issues or diagnose data or model-related problems.
- C. Correct.
Correct: Amazon SageMaker Clarify can help detect bias in the inference process, which could be a contributing factor to a drop in accuracy.
- D. Correct.
Correct: Collecting inference requests and their actual outcomes allows you to calculate key performance metrics such as accuracy, precision, recall, and helps diagnose if the issue lies in model predictions.
- E. Incorrect.
Incorrect: AWS CloudTrail primarily tracks API calls for auditing and security purposes, but it does not provide insights into model inference performance or accuracy.