MLA-C01 Question 377
Select 3You are managing a machine learning model deployed to predict customer churn for a subscription service. After monitoring the model's performance for several months, you notice a significant drop in accuracy and an increase in error rates. Upon further investigation, you find that the customer behavior patterns have shifted significantly due to a recent change in the subscription pricing model. What actions should you take to address this issue?
- A
Retrain the model using more recent data that reflects the new customer behavior patterns.
- B
Monitor for additional drift using AWS SageMaker Model Monitor to detect further changes in data distribution.
- C
Increase the frequency of model inference requests to improve accuracy.
- D
Implement data preprocessing steps to mitigate the impact of outliers caused by the pricing change.
- E
Deploy an ensemble of older and newer models to handle both historical and current customer behaviors.
Show answer and explanation
Correct answers: A, B, E
Explanation
The drop in model performance is due to concept drift, which occurs when the statistical properties of the target variable or input data change over time. To address concept drift, retraining the model with up-to-date data is crucial. Monitoring for further drift with AWS SageMaker Model Monitor can help identify new shifts in the data. Additionally, deploying an ensemble of models can be a robust strategy to handle both historical and current behavior patterns during a transitional phase.
- A. Correct.
Retraining the model with recent data is a primary step to address concept drift, as the model must learn from the new customer behavior patterns to improve performance.
- B. Correct.
AWS SageMaker Model Monitor can help detect ongoing data distribution or label distribution drift, ensuring that future shifts are identified promptly.
- C. Incorrect.
Increasing the frequency of inference requests does not address the issue of concept drift or changes in data distribution, as the model itself is outdated.
- D. Incorrect.
While preprocessing can help handle outliers, this solution does not directly address the concept drift caused by changes in customer behavior patterns due to the pricing model shift.
- E. Correct.
Deploying an ensemble of models can help mitigate the impact of both historical and current behavior patterns, ensuring better accuracy during transitional periods.