MLS-C01 Question 375
Select 3You are managing a machine learning model for predicting customer churn. The model is deployed using Amazon SageMaker, and you have implemented a retraining pipeline to maintain model performance over time. Recently, you noticed a significant drop in model accuracy due to changes in customer behavior. Which steps should you take to ensure the retraining pipeline adapts to this new data distribution?
- A
Incorporate new features into the dataset and update the feature engineering pipeline.
- B
Increase the batch size of the training job to improve generalization.
- C
Analyze the data drift metrics in Amazon SageMaker Model Monitor and adjust the retraining frequency.
- D
Update the hyperparameter tuning configuration to optimize for the new data distribution.
- E
Enable AutoML in SageMaker to automatically retrain the model based on incoming data.
Show answer and explanation
Correct answers: A, C, D
Explanation
To ensure the retraining pipeline adapts to new data distributions, it is essential to update feature engineering, monitor data drift, and adjust the retraining pipeline (e.g., retraining frequency or hyperparameter tuning). These steps ensure the model remains effective despite changes in customer behavior. Simply increasing the batch size or enabling AutoML without addressing the root cause (data distribution shift) is insufficient.
- A. Correct.
Incorporating new features and updating the feature engineering pipeline ensures the model captures relevant patterns in the new data distribution, which is crucial when data characteristics change over time.
- B. Incorrect.
Increasing the batch size does not directly address the issue of adapting to new data distributions and is unlikely to resolve the drop in model accuracy.
- C. Correct.
Analyzing data drift metrics in SageMaker Model Monitor helps detect changes in data distribution and adjusting the retraining frequency ensures the model remains up-to-date.
- D. Correct.
Updating the hyperparameter tuning configuration allows the retraining pipeline to optimize the model for the new data distribution, which is essential for maintaining performance.
- E. Incorrect.
Enabling AutoML in SageMaker is not directly relevant to the existing retraining pipeline as it does not address the specific need to adapt to the new data distribution.