MLA-C01 Question 386
Select 3You are building a machine learning model deployed on AWS to predict customer churn. After deployment, you notice a gradual degradation in model performance. Which design principles should you implement to monitor the model effectively and maintain performance over time?
- A
Set up automated data drift detection to monitor changes in input data distribution.
- B
Use Amazon SageMaker Model Monitor to track model predictions and alert on anomalies.
- C
Disable logging to minimize operational costs and simplify monitoring processes.
- D
Implement baseline metrics for model performance and track deviations over time.
- E
Rely on manual periodic reviews of model performance to identify issues.
Show answer and explanation
Correct answers: A, B, D
Explanation
Monitoring deployed machine learning models is critical to maintaining their performance over time. Automated data drift detection and tools like Amazon SageMaker Model Monitor help ensure that changes in data or model predictions are detected early. Setting baseline metrics allows for tracking deviations systematically, while relying on manual reviews or disabling logging does not align with robust monitoring principles.
- A. Correct.
Automated data drift detection is critical for identifying when the input data changes in ways that could impact model performance. This aligns with best practices for monitoring ML models.
- B. Correct.
Amazon SageMaker Model Monitor is a purpose-built tool designed to track model predictions, detect anomalies, and send alerts, making it a key component of an effective monitoring strategy.
- C. Incorrect.
Disabling logging may reduce costs but severely limits visibility into model behavior, making it a poor choice for monitoring purposes.
- D. Correct.
Establishing baseline metrics for model performance is essential because it allows you to track deviations over time and take corrective actions proactively.
- E. Incorrect.
Relying solely on manual periodic reviews is inefficient and prone to delays in detecting issues, which can lead to prolonged performance degradation.