MLA-C01 Question 374
Select 3You are a Machine Learning Engineer managing a deployed model on AWS SageMaker that predicts customer purchase likelihood. Recently, you observed a significant drop in model accuracy. Upon further investigation, you discover that customer behavior and demographics have changed significantly over time. Which actions can you take to address this issue?
- A
Monitor data drift by comparing incoming data distributions with the training dataset using Amazon SageMaker Model Monitor.
- B
Retrain the model with a new dataset that reflects the updated customer behavior and demographics.
- C
Enable SageMaker Automatic Drift Correction to automatically adjust the model's parameters to fit the new data.
- D
Implement additional feature engineering to capture the changes in customer behavior before retraining the model.
- E
Use AWS Glue to transform the incoming data to match the original training dataset distribution.
Show answer and explanation
Correct answers: A, B, D
Explanation
When model accuracy drops due to changes in input data distributions or target behavior (data drift or concept drift), it is essential to monitor the drift, retrain the model with updated data, and possibly perform additional feature engineering to account for new patterns. AWS SageMaker Model Monitor is a powerful tool for detecting drift, and retraining combined with feature engineering helps realign the model to current data. However, there is no automated mechanism in SageMaker to correct drift without human intervention, and transforming data to fit outdated patterns is counterproductive.
- A. Correct.
Monitoring data drift is a critical first step in identifying changes in input data distributions over time. Amazon SageMaker Model Monitor can help detect such drift and provide insights.
- B. Correct.
Retraining the model with a dataset that reflects the new data distribution is essential to maintain model performance, as the old model was trained on outdated patterns.
- C. Incorrect.
SageMaker does not have a feature called 'Automatic Drift Correction' for automatically adjusting model parameters. Drift must be addressed through monitoring, retraining, or updating the pipeline.
- D. Correct.
Feature engineering can help incorporate changes in customer behavior into the model, ensuring that it captures the most relevant patterns before retraining.
- E. Incorrect.
Transforming incoming data to match the original training dataset distribution is not a recommended solution, as it disregards the new patterns in the data, leading to poor model performance.