MLS-C01 Question 175
Select 3You have trained a machine learning model in Amazon SageMaker to predict customer churn. Over time, the model's accuracy decreases due to changes in customer behavior. You want to implement a scalable solution to detect and respond to model drift by automatically updating and retraining the model. Which steps should you take to achieve this?
- A
Set up Amazon SageMaker Model Monitor to detect data drift and generate alerts.
- B
Schedule periodic retraining jobs using Amazon SageMaker Pipelines with new labeled datasets.
- C
Manually review the model's performance metrics and decide when to retrain the model.
- D
Use an AWS Lambda function to automatically trigger model retraining when a drift threshold is exceeded.
- E
Deploy a new model version manually via the SageMaker console after retraining.
Show answer and explanation
Correct answers: A, B, D
Explanation
To handle model drift in a scalable and automated way, it is critical to use tools like Amazon SageMaker Model Monitor to detect drift, automate retraining workflows with SageMaker Pipelines, and trigger retraining using AWS Lambda when specific conditions are met. This approach minimizes manual intervention and ensures the model remains accurate and reliable in a dynamic environment.
- A. Correct.
Setting up Amazon SageMaker Model Monitor allows you to detect data drift or model drift by continuously monitoring the model's inputs and outputs. This is a crucial step for identifying when retraining might be necessary.
- B. Correct.
Automating retraining pipelines using Amazon SageMaker Pipelines ensures that model updates are efficient and can incorporate new labeled data regularly. This helps maintain model accuracy over time.
- C. Incorrect.
Manually reviewing performance metrics can be useful, but it is not a scalable or automated solution for responding to model drift in production environments.
- D. Correct.
AWS Lambda can be used to trigger retraining workflows by detecting drift thresholds, enabling fully automated updates to your machine learning model.
- E. Incorrect.
While manual deployment through the SageMaker console is an option, it does not align with the goal of building a scalable and automated solution for model retraining and updating.