MLA-C01 Question 368
Select 3You are a Machine Learning Engineer managing a deployed model on Amazon SageMaker that predicts customer churn. Recently, your team noticed that the model's predictions seem less accurate compared to when it was initially deployed. To monitor the model's inference, you want to detect issues such as data drift and ensure the model is performing well. Which of the following actions should you take?
- A
Enable Amazon SageMaker Model Monitor to track data quality and drift in real time.
- B
Use Amazon CloudWatch to log inference latency and system-level performance metrics.
- C
Implement a batch transform job to periodically retrain the model with fresh data.
- D
Configure baseline constraints and statistics using Amazon SageMaker Model Monitor.
- E
Deploy a new version of the model immediately to address prediction accuracy concerns.
Show answer and explanation
Correct answers: A, B, D
Explanation
Monitoring model inference involves tracking key aspects such as data quality, drift, and system performance. Amazon SageMaker Model Monitor is a powerful tool for detecting data drift and quality issues by comparing real-time inference data with a pre-defined baseline. Additionally, Amazon CloudWatch helps monitor system-level metrics like latency. These tools allow you to proactively identify and address issues affecting the model's performance. Deploying a new model or retraining should only follow after sufficient investigation and monitoring.
- A. Correct.
Correct: Amazon SageMaker Model Monitor provides capabilities to detect data drift, missing features, and other quality issues by monitoring incoming inference data against a baseline.
- B. Correct.
Correct: Amazon CloudWatch can be used to monitor system-level metrics such as inference latency, which could indicate performance bottlenecks or issues with the model.
- C. Incorrect.
Incorrect: Retraining the model is not directly related to monitoring inference. While retraining may be necessary eventually, this action does not help monitor inference and detect data drift.
- D. Correct.
Correct: Configuring baseline constraints and statistics in Amazon SageMaker Model Monitor allows you to compare real-time inference data to an established baseline, helping you detect anomalies.
- E. Incorrect.
Incorrect: Deploying a new version of the model should only be done after identifying and addressing the root cause of the issue (e.g., data drift or model degradation).