Databricks Generative AI Engineer Associate Question 277
Single answerYou are tasked with selecting an appropriate large language model (LLM) for a customer support chatbot that needs high accuracy in understanding diverse user queries, low latency for real-time responses, and manageable infrastructure costs. You have the following evaluation metrics for three candidate LLMs:
- Model A: 175B parameters, 90% accuracy, 1.5 seconds average latency, high infrastructure cost
- Model B: 13B parameters, 85% accuracy, 0.5 seconds average latency, medium infrastructure cost
- Model C: 7B parameters, 80% accuracy, 0.3 seconds average latency, low infrastructure cost
Which model should you choose given the requirements?
- A
Model A
- B
Model B
- C
Model C
- D
None of the above
Show answer and explanation
Correct answer: B
Explanation
Model B is the best choice as it strikes a balance between accuracy, latency, and cost. While Model A has the highest accuracy, its high latency and cost make it unsuitable. Model C, although cost-efficient and fast, does not meet the accuracy requirement. Therefore, Model B fulfills the requirements for the customer support chatbot most effectively.
- A. Incorrect.
Model A offers the highest accuracy but has high latency and high infrastructure costs, which do not align with the requirements for low latency and manageable costs.
- B. Correct.
Model B provides a good balance of accuracy (85%), low latency (0.5 seconds), and medium infrastructure cost, making it the best choice for the given requirements.
- C. Incorrect.
Model C has the lowest latency and cost, but its accuracy (80%) is not sufficient to meet the requirement for high accuracy in understanding diverse user queries.
- D. Incorrect.
This option is incorrect because one of the models (Model B) satisfies the requirements, so 'None of the above' is not the correct choice.