Databricks Generative AI Engineer Associate Question 158
Single answerYou are building a document retrieval system using a generative AI model with embeddings. Your source documents consist of long technical manuals averaging 10,000 words each. Expected user queries are concise (1-2 sentences) and focus on specific technical details. You aim to optimize both retrieval accuracy and computational efficiency. Which embedding model context length should you choose?
- A
Use a short context length (e.g., 128 tokens) to prioritize computational efficiency.
- B
Use a medium context length (e.g., 512 tokens) to balance document coverage and query relevance.
- C
Use a long context length (e.g., 4,096 tokens) to ensure the entire document is captured in a single embedding.
- D
Use the maximum context length supported by the model to include as much document information as possible.
Show answer and explanation
Correct answer: B
Explanation
For optimizing retrieval accuracy and computational efficiency when dealing with long source documents and concise queries, a medium context length (e.g., 512 tokens) ensures enough document context is embedded to answer specific queries effectively, while avoiding excessive computational costs associated with longer context lengths.
- A. Incorrect.
A short context length (e.g., 128 tokens) would severely limit the amount of document information included in the embedding, reducing retrieval accuracy, especially given the long source documents.
- B. Correct.
A medium context length (e.g., 512 tokens) provides a balance between computational cost and the ability to capture sufficient context from the source documents to respond to concise queries effectively.
- C. Incorrect.
A long context length (e.g., 4,096 tokens) would unnecessarily increase computational costs and is not optimal for concise user queries. It might also lead to inefficiencies in processing the large technical manuals.
- D. Incorrect.
Using the maximum context length supported by the model might seem beneficial but would result in excessive computational overhead without significantly improving retrieval accuracy for short queries.