MLA-C01 Question 269
Single answerYou are deploying a machine learning model using Amazon SageMaker. The model will be used to process thousands of incoming requests per second with low latency requirements. Which type of deployment endpoint should you choose to meet these requirements?
- A
Serverless endpoint
- B
Real-time endpoint
- C
Asynchronous endpoint
- D
Batch inference endpoint
Show answer and explanation
Correct answer: B
Explanation
Real-time endpoints in Amazon SageMaker are the best choice for scenarios requiring low-latency and high-throughput inference. Unlike serverless or asynchronous endpoints, which are better suited to intermittent or non-real-time workloads, real-time endpoints keep a fleet of instances active to handle high volumes of requests instantly. Batch inference is not applicable to real-time processing as it is meant for offline use cases.
- A. Incorrect.
Serverless endpoints are designed for applications with intermittent traffic, where you pay only for the compute time used. They may not be suitable for high-throughput, low-latency use cases like this.
- B. Correct.
Real-time endpoints are specifically designed for low-latency, high-throughput scenarios. They maintain an active fleet of instances to handle incoming requests instantly.
- C. Incorrect.
Asynchronous endpoints are optimized for large-scale inference workloads where real-time latency is not a requirement. They are not ideal for processing thousands of requests per second with low latency.
- D. Incorrect.
Batch inference endpoints are used for offline or large-scale inference jobs on datasets that are already stored in Amazon S3. They are not applicable for real-time, high-throughput use cases.