Databricks Generative AI Engineer Associate Question 281
Single answerYour team is tasked with selecting an appropriate large language model (LLM) for a customer support chatbot. You have access to quantitative evaluation metrics for several models, including perplexity, latency, and cost. The team wants a model that balances low perplexity (to ensure high-quality responses) and low latency (to ensure fast responses) while staying within the budget. Which LLM would be the best choice based on the following metrics?
Model A: Perplexity = 12, Latency = 500ms, Cost = $0.10/request Model B: Perplexity = 10, Latency = 800ms, Cost = $0.08/request Model C: Perplexity = 15, Latency = 400ms, Cost = $0.12/request Model D: Perplexity = 11, Latency = 600ms, Cost = $0.09/request
- A
Model A
- B
Model B
- C
Model C
- D
Model D
Show answer and explanation
Correct answer: D
Explanation
When selecting an LLM, it's important to balance multiple quantitative evaluation metrics based on the use case. In this scenario, Model D offers the best trade-off between perplexity, latency, and cost, making it ideal for a real-time customer support chatbot. While other models excel in specific metrics, they fail to meet the overall requirements as effectively as Model D.
- A. Incorrect.
Model A has a reasonable perplexity and latency, but its cost is higher than Model D while offering only slightly worse perplexity and latency performance.
- B. Incorrect.
Model B has the lowest perplexity and cost, but its high latency of 800ms makes it less suitable for real-time customer support, where faster response times are critical.
- C. Incorrect.
Model C has the lowest latency, but its perplexity is the highest, which may result in lower-quality responses. Additionally, its cost is the highest among all models.
- D. Correct.
Model D provides a balance between low perplexity, acceptable latency (600ms), and cost ($0.09/request), making it the best overall choice for the given constraints.