MLA-C01 Question 384
Select 3A company has deployed a machine learning model into production using Amazon SageMaker. The model's performance has begun to degrade over time due to changes in the input data distribution. As a certified AWS Machine Learning Engineer, which monitoring practices should you implement to ensure the model remains reliable and performant?
- A
Set up Amazon SageMaker Model Monitor to track data quality and drift over time.
- B
Enable automatic hyperparameter tuning in SageMaker to continuously optimize the model.
- C
Establish baseline metrics for data quality and model predictions using historical data.
- D
Configure Amazon CloudWatch alarms to detect anomalies in key model performance metrics.
- E
Periodically retrain the model using updated training data without evaluating its performance.
Show answer and explanation
Correct answers: A, C, D
Explanation
Monitoring deployed machine learning models is critical to ensure they remain reliable and performant under changing conditions. Using tools like SageMaker Model Monitor allows for automated data quality and drift detection. Establishing baseline metrics ensures you have a reference to measure deviations, while CloudWatch alarms provide real-time notifications for anomalies. These practices collectively ensure the model is monitored effectively and potential issues are identified and resolved in a timely manner.
- A. Correct.
Amazon SageMaker Model Monitor is specifically designed to track data quality, data drift, and other important metrics over time, making it essential for monitoring deployed models.
- B. Incorrect.
Automatic hyperparameter tuning is useful during the training phase, but it is not a monitoring practice and does not directly address the issue of performance degradation in deployed models.
- C. Correct.
Establishing baseline metrics for data quality and predictions is a critical step in monitoring, as it provides a reference point to compare ongoing model performance.
- D. Correct.
CloudWatch alarms can help detect anomalies in key performance metrics such as latency, accuracy, or error rates, making it an important component of a monitoring strategy.
- E. Incorrect.
Retraining the model without evaluating its performance is not a recommended monitoring practice, as it introduces the risk of deploying an ineffective or harmful model.