NCA-GENL Question 148
Select 3You are tasked with building a generative AI pipeline that involves fine-tuning a large language model (LLM) on a proprietary dataset. After deploying the model, the team notices an increase in latency and a decrease in response quality. Which of the following actions should you prioritize to resolve these issues?
- A
Optimize the model architecture by reducing the number of layers.
- B
Evaluate the quality of the proprietary dataset for noise or inconsistencies.
- C
Use quantization techniques to reduce the model size without significantly impacting accuracy.
- D
Implement a caching mechanism to store frequent responses and reduce compute load.
- E
Increase the training epochs to further fine-tune the model for better performance.
Show answer and explanation
Correct answers: B, C, D
Explanation
The best course of action is a combination of evaluating the dataset for quality (to address response degradation), applying quantization techniques to reduce latency, and using caching mechanisms to optimize response times. Reducing model layers might compromise response quality, and increasing epochs does not directly address latency concerns.
- A. Incorrect.
Optimizing the model architecture by reducing the number of layers can decrease latency but may lead to a significant loss in the model's capability to generate high-quality responses, making this an unsuitable first step.
- B. Correct.
Evaluating the proprietary dataset is critical as noisy or inconsistent data could directly impact the model's ability to generate high-quality responses.
- C. Correct.
Quantization techniques can help reduce the model size, which is beneficial for decreasing latency without heavily degrading performance, making it a practical solution.
- D. Correct.
Implementing a caching mechanism can reduce the computational load by reusing previously generated responses, effectively addressing latency issues without impacting quality.
- E. Incorrect.
While increasing the training epochs might improve response quality, it does not address the latency issue and could lead to overfitting if not handled carefully.