NCA-GENL Question 171
Select 4A company wants to deploy a generative AI large language model (LLM) to serve customer queries in real-time. The expected workload involves processing thousands of concurrent queries with minimal latency and high accuracy. Which combination of system components is most suitable to meet these requirements?
- A
High-performance GPUs optimized for inferencing tasks
- B
A CPU-only infrastructure with multi-core processors
- C
NVIDIA TensorRT for optimized model inference
- D
NVMe-based high-speed storage for faster data access
- E
Cloud-based deployment with auto-scaling capabilities
Show answer and explanation
Correct answers: A, C, D, E
Explanation
To meet the requirements of real-time, high-concurrency LLM deployment, a combination of high-performance GPUs, optimized inference software (like NVIDIA TensorRT), high-speed storage (like NVMe), and a scalable cloud infrastructure is necessary. CPUs alone are insufficient for such heavy workloads, as they lack the specialized capabilities required for LLM inferencing at scale.
- A. Correct.
High-performance GPUs, such as NVIDIA GPUs, are specialized for handling inferencing tasks efficiently, offering low latency and high throughput for LLM workloads.
- B. Incorrect.
While CPUs are versatile, they are not optimized for the computational demands of LLM inference, especially for real-time processing of thousands of queries.
- C. Correct.
NVIDIA TensorRT is a software library that accelerates inference, reducing latency and improving performance, making it critical for real-time LLM applications.
- D. Correct.
NVMe-based high-speed storage ensures that data required for inference is accessed quickly, minimizing bottlenecks in the overall system performance.
- E. Correct.
Cloud-based deployment with auto-scaling ensures that the system can handle spikes in workload dynamically, which is essential for unpredictable, high-concurrency scenarios.