MLS-C01 Question 174
Select 3You are managing a machine learning model deployed on Amazon SageMaker that predicts customer churn. Over the past month, the model's performance has degraded due to changes in customer behavior. You need to update and retrain the model to improve accuracy. Which steps should you take to address this issue?
- A
Collect new labeled data reflecting recent customer behavior and add it to the training dataset.
- B
Use Amazon SageMaker Model Monitor to detect data drift and automatically update the model without retraining it.
- C
Retrain the model using the updated training dataset and optimize hyperparameters using SageMaker Automatic Model Tuning.
- D
Archive the old model and deploy the retrained model to SageMaker endpoints.
- E
Manually adjust the model's weights to account for changes in customer behavior without retraining.
Show answer and explanation
Correct answers: A, C, D
Explanation
When a model's performance degrades due to changes in input data or target patterns, it is essential to retrain it with updated, relevant data. Collecting new labeled data, retraining the model while optimizing hyperparameters, and deploying the new model after archiving the old one are all critical steps to maintain and improve model performance. While tools like SageMaker Model Monitor can alert you to data drift, they do not replace the need for retraining. Manual weight adjustments are neither practical nor effective for addressing performance issues at scale.
- A. Correct.
Collecting new labeled data reflecting recent customer behavior is crucial to ensure the model is trained on the most relevant data and can adapt to changes in patterns.
- B. Incorrect.
Amazon SageMaker Model Monitor can detect data drift, but it does not automatically update or retrain models. The retraining process must still be done manually or triggered programmatically.
- C. Correct.
Retraining the model with updated data and optimizing hyperparameters using SageMaker Automatic Model Tuning is essential for improving the model's performance.
- D. Correct.
Archiving the old model and deploying the retrained model ensures you have a backup and can seamlessly replace the underperforming model in production.
- E. Incorrect.
Manually adjusting the model's weights is not a recommended or scalable approach, as it does not leverage the historical data or retrain the model properly.