Databricks Generative AI Engineer Associate Question 279
Single answerYour team is tasked with selecting a Large Language Model (LLM) for a customer service chatbot that must handle domain-specific queries efficiently while keeping latency low. You have access to the following evaluation metrics for two candidate models: Model A (7B parameters, GPT-based architecture) and Model B (13B parameters, Transformer-XL architecture). The evaluation results indicate:
- Model A: Higher accuracy on domain-specific tasks (85%), lower latency (100ms)
- Model B: Lower accuracy on domain-specific tasks (78%), higher latency (180ms)
Given these metrics, which model would be the most appropriate choice for your use case?
- A
Model A, because it has higher accuracy and lower latency, making it better suited for customer service queries.
- B
Model A, because smaller models generally perform better in real-time applications regardless of accuracy.
- C
Model B, because it has a larger parameter size and may generalize better to new queries.
- D
Model B, because Transformer-XL architecture is inherently better for domain-specific tasks.
Show answer and explanation
Correct answer: A
Explanation
For a customer service chatbot, both accuracy and latency are critical. Model A outperforms Model B in these metrics, with higher accuracy on domain-specific tasks and significantly lower latency, making it the most appropriate choice for the use case.
- A. Correct.
Model A, with higher accuracy and lower latency, is the better choice for this use case. Customer service tasks benefit from models that can deliver accurate domain-specific results quickly.
- B. Incorrect.
While smaller models can have advantages in real-time applications, the decision should not disregard accuracy, which is critical for customer service tasks.
- C. Incorrect.
A larger parameter size does not guarantee better performance, especially as Model B has lower accuracy on the domain-specific task and higher latency, which are critical factors in this scenario.
- D. Incorrect.
There is no evidence that Transformer-XL architecture is inherently better for domain-specific tasks, and the evaluation metrics show that Model B underperforms in this context.