MLA-C01 Question 400
Select 4A data science team at an e-commerce company is running multiple machine learning models on Amazon SageMaker to provide real-time product recommendations. They are noticing intermittent delays in predictions during peak traffic hours. As an AWS Certified Machine Learning Engineer, which key performance metrics should you evaluate to identify and address the bottleneck in their ML infrastructure?
- A
Instance utilization rates for endpoints
- B
Model prediction accuracy on test datasets
- C
Throughput of the endpoint under varying traffic loads
- D
Endpoint availability during peak traffic times
- E
Scalability of the deployed endpoint under increased traffic
- F
Data preprocessing latency in the pipeline
Show answer and explanation
Correct answers: A, C, D, E
Explanation
To address delays in real-time predictions during peak traffic, you need to diagnose key performance metrics related to the ML infrastructure, such as instance utilization, throughput, availability, and scalability. These metrics help identify whether the infrastructure is adequately provisioned and capable of handling the traffic load. Accuracy and preprocessing latency, while important, are not directly related to infrastructure performance in this scenario.
- A. Correct.
Instance utilization rates for endpoints are crucial to understanding whether the infrastructure is over or under-utilized, which could lead to delays or inefficiency.
- B. Incorrect.
Model prediction accuracy on test datasets is unrelated to infrastructure performance and does not directly address delays during real-time predictions.
- C. Correct.
Throughput measures how many predictions the endpoint can handle per second and is directly relevant to identifying performance bottlenecks under load.
- D. Correct.
Endpoint availability ensures that the endpoint remains functional during high traffic, a key metric for diagnosing performance issues.
- E. Correct.
Scalability evaluates how well the endpoint can handle increased traffic, which is critical for resolving delays during peak hours.
- F. Incorrect.
Data preprocessing latency is important for pipeline performance but is not specific to real-time endpoint delays during prediction.