NCA-GENL Question 172
Select 3A company wants to deploy a generative AI Large Language Model (LLM) for real-time customer support. They require a system capable of handling high inference throughput and low latency for user queries. Which combination of system components would best meet these requirements?
- A
High-performance GPUs designed for inferencing workloads
- B
Large-capacity, high-latency hard disk drives (HDDs)
- C
Low-latency, high-bandwidth memory (e.g., HBM or GDDR6)
- D
Optimized software frameworks such as NVIDIA TensorRT
- E
General-purpose CPUs without hardware acceleration
Show answer and explanation
Correct answers: A, C, D
Explanation
Deploying a generative AI LLM for real-time customer support requires hardware and software components that can handle the high computational demands of inference with low latency. High-performance GPUs, low-latency memory, and optimized software frameworks like NVIDIA TensorRT work together to maximize throughput and minimize inference time. HDDs and general-purpose CPUs without acceleration are unsuitable for such workloads due to their limitations in speed and scalability.
- A. Correct.
High-performance GPUs are essential for handling the computational demands of LLM inference, especially for real-time applications requiring high throughput and low latency.
- B. Incorrect.
High-latency hard disk drives (HDDs) are not suitable for real-time inference tasks, as they do not provide the speed required for high-performance workloads. Solid-state drives (SSDs) or in-memory solutions are preferred.
- C. Correct.
Low-latency, high-bandwidth memory (e.g., HBM or GDDR6) is crucial for ensuring that the GPU can access data quickly, reducing bottlenecks during inference.
- D. Correct.
Optimized software frameworks such as NVIDIA TensorRT are specifically designed to accelerate inference performance on NVIDIA GPUs, making them an ideal choice for real-time LLM deployments.
- E. Incorrect.
General-purpose CPUs without hardware acceleration cannot efficiently handle the computational demands of LLM inference at scale, resulting in poor performance and higher latency.