MLA-C01 Question 365
Select 2You deployed a machine learning model to Amazon SageMaker and configured it to handle real-time inference requests. The model’s predictions are being monitored by stakeholders for accuracy and consistency. Recently, stakeholders reported a decline in prediction quality. Upon investigation, you suspect that the input data distribution has shifted compared to the training data distribution. What steps should you take to monitor and address this issue effectively?
- A
Enable Amazon SageMaker Model Monitor to track data drift on input features and compare them to the training baseline.
- B
Retrain the model immediately using the latest inference data without analyzing the data drift.
- C
Use Amazon CloudWatch Logs to capture real-time inference requests and manually analyze input distributions.
- D
Generate and upload a baseline dataset to Amazon S3 that represents the original training data, then configure SageMaker Model Monitor to detect data drift.
- E
Set up Amazon SageMaker Clarify to monitor bias in real-time predictions as a proxy for data drift.
Show answer and explanation
Correct answers: A, D
Explanation
To effectively monitor model inference and address issues like data drift, you should use Amazon SageMaker Model Monitor. It allows you to compare incoming inference data distributions to a baseline dataset generated from the training data. By detecting data drift, you can determine whether the input data has changed significantly, which may explain the decline in prediction quality. Retraining the model or addressing other issues should only occur after understanding data drift patterns. While Amazon CloudWatch Logs and SageMaker Clarify are useful tools, they do not specifically address data drift monitoring in this scenario.
- A. Correct.
This is correct. Amazon SageMaker Model Monitor can be used to track data drift by comparing incoming inference data to a baseline dataset created from the training data.
- B. Incorrect.
This is incorrect. While retraining the model may eventually be necessary, doing so without analyzing the data drift or understanding the root cause is not a best practice.
- C. Incorrect.
This is incorrect. Although Amazon CloudWatch Logs can capture inference requests, analyzing input distributions manually is not efficient or recommended compared to using SageMaker Model Monitor.
- D. Correct.
This is correct. Creating a baseline dataset from the training data and using SageMaker Model Monitor to evaluate incoming data against the baseline is a recommended approach to monitor for data drift.
- E. Incorrect.
This is incorrect. Amazon SageMaker Clarify is designed to detect bias in models and data but is not specifically used to monitor data drift.