Databricks Generative AI Engineer Associate Question 160
Single answerYou are designing a Generative AI application on Databricks that will use an embedding model to retrieve relevant information from a collection of long legal documents. The source documents average 20,000 tokens each, and user queries are typically concise, averaging 50 tokens. Your optimization strategy prioritizes retrieval accuracy while minimizing computational costs. What is the most appropriate embedding model context length to use?
- A
A context length that matches the average length of the source documents (20,000 tokens).
- B
A context length that matches the average length of the user queries (50 tokens).
- C
A context length that accommodates the document segments likely to contain relevant context (e.g., 2,000 tokens).
- D
A context length that is the maximum supported by the embedding model, regardless of data characteristics.
Show answer and explanation
Correct answer: C
Explanation
The optimal context length for an embedding model depends on the source document size, query length, and optimization strategy. Since the goal is to maximize retrieval accuracy while minimizing costs, breaking the long documents into smaller, meaningful segments (e.g., 2,000 tokens) ensures the embedding model focuses on relevant information without excessive computational expense.
- A. Incorrect.
Using a context length of 20,000 tokens would significantly increase computational costs and is unnecessary for identifying relevant segments of the documents.
- B. Incorrect.
A context length of 50 tokens is too short to capture sufficient context from the source documents, leading to poor retrieval accuracy.
- C. Correct.
A context length of 2,000 tokens strikes a balance between capturing enough context to improve retrieval accuracy and keeping computational costs manageable. It allows for breaking the documents into smaller, meaningful segments.
- D. Incorrect.
Using the maximum supported context length regardless of data characteristics may unnecessarily increase computational costs without a tangible improvement in retrieval accuracy.