Databricks Generative AI Engineer Associate Question 157
Single answerYou are designing a generative AI solution that uses an embedding model to process customer support documents and answer user queries. Each document is approximately 2,000 tokens long, and user queries are typically 10-20 tokens. Your goal is to optimize for accuracy while minimizing computational costs. Which embedding model context length should you select?
- A
512 tokens
- B
1,024 tokens
- C
2,048 tokens
- D
4,096 tokens
Show answer and explanation
Correct answer: C
Explanation
The context length of an embedding model should be chosen based on the size of the source documents, the expected length of queries, and the need to optimize for accuracy and computational cost. Since the documents are 2,000 tokens long, using a context length of 2,048 tokens ensures that the entire document is processed. Choosing a smaller context length would result in incomplete representations, while a much larger context length would unnecessarily increase computational costs.
- A. Incorrect.
A context length of 512 tokens would be insufficient to represent the full content of the 2,000-token documents, resulting in incomplete embeddings and reduced accuracy.
- B. Incorrect.
A context length of 1,024 tokens improves over 512 tokens but still falls short of capturing the entire document, which impacts the quality of embeddings for long documents.
- C. Correct.
A context length of 2,048 tokens matches the length of the source documents, allowing the embedding model to process the entire document for maximum accuracy without unnecessary computational overhead.
- D. Incorrect.
A context length of 4,096 tokens exceeds the length of the documents, leading to increased computational costs without adding any accuracy benefits since the documents are only 2,000 tokens long.