Databricks Generative AI Engineer Associate Question 161
Single answerYou are developing a generative AI solution for a legal firm to assist with document search and summarization. The source documents include lengthy contracts averaging 10,000 tokens, while user queries typically contain 20-50 tokens. Your optimization strategy prioritizes accurate context retrieval over model performance speed. Which embedding model context length should you choose?
- A
256 tokens
- B
512 tokens
- C
2048 tokens
- D
8192 tokens
Show answer and explanation
Correct answer: C
Explanation
The selection of an embedding model context length depends on the size of the source documents, the typical query length, and the optimization strategy. In this scenario, the source documents are very large, requiring a sufficiently long context length to capture meaningful information. A 2048-token context strikes a balance between capturing enough of the document and maintaining computational efficiency, aligning with the optimization strategy of prioritizing accurate context retrieval.
- A. Incorrect.
A context length of 256 tokens is too short to capture the necessary context from lengthy documents like legal contracts, which average 10,000 tokens. This choice would result in incomplete or inaccurate embeddings for retrieval.
- B. Incorrect.
A context length of 512 tokens is still too short for the given source documents. While it provides slightly more context than 256 tokens, it is unlikely to retrieve enough meaningful data from such large documents.
- C. Correct.
A context length of 2048 tokens is a suitable choice for balancing the large size of the source documents and the goal of maintaining accurate context retrieval. It allows for a significant portion of the document to be embedded while remaining computationally feasible.
- D. Incorrect.
A context length of 8192 tokens is unnecessarily large for this scenario. Although it could theoretically capture more of the document, it would be computationally expensive and may exceed the optimization needs for accurate retrieval.