NCA-GENL Question 152
Select 3You are assisting in the deployment of a large language model (LLM) for a real-time customer support application under the supervision of a senior team member. During the evaluation phase, the model experiences increased latency as the number of concurrent users grows. Which actions should you prioritize to address this issue?
- A
Optimize the model parameters to reduce computational overhead during inference.
- B
Scale the deployment infrastructure horizontally by adding more GPU-enabled instances.
- C
Switch to a smaller pre-trained model without considering the application's performance requirements.
- D
Implement caching mechanisms for frequently repeated user queries to reduce redundant computations.
- E
Disable monitoring tools to free up resources for model inference.
Show answer and explanation
Correct answers: A, B, D
Explanation
To address the latency issue in a scalable and reliable manner, actions should focus on optimizing the model's efficiency, scaling infrastructure to handle increased demand, and implementing mechanisms like caching to improve response times. These steps ensure that the deployment can sustain performance under growing user load while maintaining reliability and quality.
- A. Correct.
Optimizing model parameters can help reduce computational overhead and improve inference speed, making it a valid action to address latency issues.
- B. Correct.
Scaling the deployment infrastructure horizontally ensures that additional computational resources are available to handle the increased number of users, reducing latency.
- C. Incorrect.
Switching to a smaller model without assessing the application's performance requirements could degrade the quality of responses, making this an inappropriate choice without further analysis.
- D. Correct.
Caching frequently repeated queries reduces redundant computations and enhances performance, making it an effective solution for addressing latency in high-load scenarios.
- E. Incorrect.
Disabling monitoring tools would negatively impact the ability to identify and diagnose issues, making this an inappropriate action.