MLA-C01 Question 398
Select 3You are managing a machine learning inference workload hosted on Amazon SageMaker endpoints. A sudden spike in user traffic has caused the latency of your predictions to increase significantly. Your goal is to maintain low latency while ensuring the system remains cost-effective. Which key performance metrics should you monitor and optimize to address this issue?
- A
Throughput of the SageMaker endpoint
- B
Utilization of the underlying instance resources
- C
Fault tolerance of the endpoint
- D
Scalability of the endpoint
- E
Availability of the endpoint
Show answer and explanation
Correct answers: A, B, D
Explanation
To address increased latency during a traffic spike, you need to monitor and optimize throughput, utilization, and scalability. Throughput ensures the system can handle the incoming requests, utilization helps identify if resources are being overused, and scalability ensures the system can adjust to higher demand. Fault tolerance and availability are important metrics for system reliability but are not directly relevant to optimizing latency in this scenario.
- A. Correct.
Throughput is critical to measure how many requests the endpoint can process per second. A spike in traffic may overwhelm the current throughput capacity, leading to latency issues.
- B. Correct.
Monitoring utilization of resources like CPU, GPU, and memory is essential to determine if the current instances can handle the increased load or need to be scaled up or out.
- C. Incorrect.
Fault tolerance ensures the system can recover from failures, but it does not directly address latency caused by a traffic spike.
- D. Correct.
Scalability is crucial in addressing increased demand. It allows the system to automatically adjust resources to maintain low latency during traffic spikes.
- E. Incorrect.
Availability measures whether the endpoint is reachable, but it does not address performance issues like latency increases caused by traffic surges.