MLA-C01 Question 381
Select 2You are a Machine Learning Engineer at a financial services company using an Amazon SageMaker endpoint to serve a trained model for predicting loan defaults. The model performance has recently degraded due to changes in customer behavior, and you suspect data drift in the input features. Which techniques can you use to monitor data quality and model performance effectively?
- A
Enable Amazon SageMaker Model Monitor to continuously analyze the incoming features against a baseline dataset.
- B
Set up Amazon CloudWatch alarms to trigger notifications based on model prediction latency metrics.
- C
Use Amazon SageMaker Clarify to identify bias in predictions and monitor for fairness issues.
- D
Configure Amazon SageMaker Model Monitor to track prediction quality by comparing predictions with actual labels.
- E
Implement a custom Lambda function to periodically retrain the model with fresh data.
Show answer and explanation
Correct answers: A, D
Explanation
Monitoring data quality and model performance is critical for maintaining reliable predictions, especially in production systems. SageMaker Model Monitor provides built-in capabilities to detect data drift and evaluate prediction quality, making it a powerful tool for these scenarios. Other techniques like CloudWatch or SageMaker Clarify serve different purposes and are not directly related to data quality or performance monitoring.
- A. Correct.
Correct: Amazon SageMaker Model Monitor can analyze incoming features and compare them to a baseline dataset to detect data drift, ensuring input data quality.
- B. Incorrect.
Incorrect: While CloudWatch can monitor operational metrics like model latency, it does not directly monitor data quality or model performance.
- C. Incorrect.
Incorrect: SageMaker Clarify is primarily used for bias detection and explainability, not for monitoring data quality or model performance over time.
- D. Correct.
Correct: SageMaker Model Monitor can track prediction quality by comparing model predictions with actual labels, helping to evaluate performance.
- E. Incorrect.
Incorrect: While retraining the model with fresh data is a valid practice, it is not a monitoring technique and does not directly address data or model performance monitoring.