NCA-GENL Question 170
Select 3A data science team is deploying a large language model (LLM) for real-time customer support. The model requires low latency responses, high computational performance, and efficient data handling for large-scale text inputs. Which combination of system components should be prioritized to meet these requirements?
- A
High-performance GPUs with sufficient tensor cores for accelerated computations
- B
Solid-state drives (SSDs) to ensure fast data access and model loading
- C
High-capacity hard disk drives (HDDs) for cost-effective storage
- D
A high-speed networking setup for fast communication between distributed nodes
- E
Increased CPU core count for better single-threaded performance
Show answer and explanation
Correct answers: A, B, D
Explanation
To deploy a large language model for real-time customer support, the system needs components that optimize computational performance, memory access, and communication. High-performance GPUs enable fast inference, SSDs ensure quick data access, and high-speed networking reduces latency in distributed setups. While HDDs and increased CPU cores have their uses, they are not as critical for this specific scenario.
- A. Correct.
High-performance GPUs with tensor cores are essential for the efficient training and inference of large language models. They significantly reduce latency by accelerating matrix operations and neural computations.
- B. Correct.
SSDs provide faster read and write speeds compared to HDDs, enabling quicker loading of large model weights and datasets, which is crucial for real-time applications.
- C. Incorrect.
While HDDs are cost-effective, they are slower than SSDs and are not well-suited for real-time applications requiring high-speed data access.
- D. Correct.
A high-speed networking setup is necessary for distributed systems to communicate efficiently, particularly when the model is deployed across multiple nodes.
- E. Incorrect.
Increased CPU core count is beneficial for multitasking, but LLMs primarily benefit from GPU acceleration rather than CPU performance, especially for intensive computations.