MLS-C01 Question 382
Select 3Your team has deployed a machine learning model for real-time predictions using Amazon SageMaker. After a few weeks of deployment, you notice a significant drop in the model's prediction accuracy. Upon investigation, you suspect data drift might be the cause. Which steps should you take to detect and mitigate the drop in performance?
- A
Set up Amazon SageMaker Model Monitor to track data drift and alert on anomalies.
- B
Retrain the model on the initial training dataset to restore its original accuracy.
- C
Implement Amazon CloudWatch alarms to monitor model latency and throughput.
- D
Collect recent input data and ground truth labels, then validate the model's predictions.
- E
Use Amazon SageMaker Clarify to analyze feature importance and detect changes in feature distributions.
Show answer and explanation
Correct answers: A, D, E
Explanation
A drop in model performance can often be attributed to data drift, where the input data distribution changes over time. To detect and mitigate this, you should use tools like Amazon SageMaker Model Monitor to track data drift and analyze input features. Additionally, validating the model's predictions with recent data and ground truth labels helps confirm the root cause of the issue. SageMaker Clarify can provide valuable insights into changes in feature distributions, which is critical for addressing performance drops caused by data drift.
- A. Correct.
Setting up Amazon SageMaker Model Monitor is an effective way to track data drift and detect anomalies in the input data compared to the model's training data.
- B. Incorrect.
Retraining the model on the initial training dataset will not address the root cause (data drift) and may not improve performance. A new dataset reflecting the current data distribution is required.
- C. Incorrect.
Monitoring model latency and throughput using Amazon CloudWatch alarms is useful for detecting performance issues related to system infrastructure but does not directly address data drift.
- D. Correct.
Collecting recent input data and ground truth labels allows you to validate the model's predictions and identify whether the drop in accuracy is due to data drift or other issues.
- E. Correct.
Amazon SageMaker Clarify can help analyze feature importance and detect changes in the distribution of input features, which is critical for understanding potential causes of data drift.