MLA-C01 Question 246
Select 3A financial services company uses Amazon SageMaker to train a machine learning model that predicts credit card fraud. The model is deployed as a real-time endpoint. After deployment, the data science team notices that the model's performance on new data has degraded over time. Which set of actions should the team take to address this issue?
- A
Enable Amazon SageMaker Model Monitor to analyze real-time endpoint data for data drift.
- B
Retrain the model using the original training dataset to regain the initial performance.
- C
Collect new labeled data, retrain the model with this data, and deploy the updated model.
- D
Use Amazon SageMaker Ground Truth to label a new dataset and include it in the training pipeline.
- E
Reduce the size of the input feature space to minimize overfitting issues.
Show answer and explanation
Correct answers: A, C, D
Explanation
The model's degraded performance is likely due to data drift, where the distribution of new data differs from the training data. Addressing this issue involves detecting data drift using tools like Amazon SageMaker Model Monitor, collecting new labeled data to reflect the updated distribution, and retraining the model with the new data. Amazon SageMaker Ground Truth can assist in creating high-quality labeled datasets for retraining. Simply retraining with the old data or reducing the feature space would not solve the problem as they do not address the shift in data distribution.
- A. Correct.
Enabling Amazon SageMaker Model Monitor helps detect data drift by continuously analyzing input data and comparing it to the original training data distribution. This step is essential for identifying the root cause of model performance degradation.
- B. Incorrect.
Retraining the model using the original training dataset is not a suitable solution because the issue arises from the new data, which may have shifted distributions compared to the training data. This would not address the root cause of the degradation.
- C. Correct.
Collecting new labeled data that reflects the current data distribution and retraining the model with it ensures that the model adapts to the new data patterns. This is a critical step in resolving performance degradation.
- D. Correct.
Using Amazon SageMaker Ground Truth to label a new dataset helps create high-quality labeled data that can be used to retrain the model, making it more aligned with current data distributions.
- E. Incorrect.
Reducing the size of the input feature space is not relevant in this context because the issue is related to data drift and not overfitting due to too many input features.