MLS-C01 Question 387
Select 3You have deployed a machine learning model using Amazon SageMaker and are monitoring its performance in production. After observing a decline in model accuracy, you suspect there may be data drift in the input features. Which of the following actions should you take to effectively monitor and address this issue?
- A
Enable Amazon SageMaker Model Monitor to analyze feature distributions and detect data drift.
- B
Retrain the model immediately using the same training dataset to address the performance decline.
- C
Set up a schedule in Amazon SageMaker Model Monitor to capture and analyze inference request data.
- D
Use Amazon SageMaker Clarify to assess feature importance and check for potential bias in the data.
- E
Configure custom CloudWatch alarms to notify you when key metrics, such as accuracy, fall below a defined threshold.
Show answer and explanation
Correct answers: A, C, E
Explanation
Monitoring the performance of a deployed machine learning model is crucial to maintaining its effectiveness in production. Amazon SageMaker Model Monitor is specifically designed to detect issues like data drift by analyzing feature distributions and comparing them to a baseline. Additionally, setting up CloudWatch alarms ensures you are alerted when metrics like accuracy degrade. Retraining the model should only be considered after analyzing and addressing the underlying issue, such as data drift.
- A. Correct.
Correct: Amazon SageMaker Model Monitor can be enabled to continuously analyze the input data and detect any data drift by comparing the current dataset to the baseline. This is an essential step to monitor changes in feature distributions.
- B. Incorrect.
Incorrect: Retraining the model without addressing the root cause of the performance decline (e.g., data drift) may not improve the model's performance. You need to first understand the nature of the data drift before retraining.
- C. Correct.
Correct: Scheduling regular data capture and analysis in Amazon SageMaker Model Monitor allows for systematic monitoring of inference data over time to detect issues like data drift.
- D. Incorrect.
Incorrect: While Amazon SageMaker Clarify is a great tool to analyze bias and understand feature importance, it is not directly designed to monitor data drift in production.
- E. Correct.
Correct: Setting up CloudWatch alarms for key metrics such as model accuracy or latency allows you to be notified promptly when the model's performance declines, enabling quicker response times.