MLA-C01 Question 387
Select 3You are a Machine Learning Engineer responsible for deploying and monitoring a fraud detection model on AWS. The model's predictions are used to make real-time decisions, and ensuring high accuracy and reliability is critical for the business. Which design principles should you prioritize to ensure effective monitoring of this ML system?
- A
Set up automated alerts for deviations in model performance metrics, such as accuracy or precision.
- B
Continuously monitor data drift and retrain the model only if the drift exceeds a predefined threshold.
- C
Focus solely on infrastructure-level monitoring, such as CPU and memory usage, as it ensures application uptime.
- D
Implement monitoring for both input data quality and output predictions to detect anomalies.
- E
Rely on manual monitoring by data scientists to identify issues in the system.
Show answer and explanation
Correct answers: A, B, D
Explanation
Effective monitoring of ML systems involves a combination of approaches, including tracking model performance metrics, monitoring for data drift, and ensuring input data quality and output prediction consistency. Automated alerts and proactive retraining strategies are essential for maintaining the reliability of real-time systems. Focusing only on infrastructure or manual monitoring is insufficient for addressing ML-specific challenges.
- A. Correct.
Correct: Automated alerts for deviations in model performance metrics help quickly identify and address performance degradation, which is critical for ML systems in real-time use cases.
- B. Correct.
Correct: Monitoring data drift and retraining the model when needed ensures the model remains effective as input data distributions change over time.
- C. Incorrect.
Incorrect: While infrastructure-level monitoring is important, it does not address ML-specific issues like data drift, model bias, or prediction anomalies.
- D. Correct.
Correct: Monitoring both input data quality and model predictions ensures the system can detect anomalies that may affect performance or reliability.
- E. Incorrect.
Incorrect: Relying solely on manual monitoring is not scalable or efficient for real-time systems, and it increases the risk of delayed issue detection.