Databricks Generative AI Engineer Associate Question 159
Single answerYou are designing a semantic search solution using a generative AI model in Databricks. Your source documents are relatively short, averaging 500 tokens each, and the expected queries are concise, typically less than 30 tokens. Your optimization strategy is to minimize computational costs without losing relevance in the embeddings. Which embedding model context length should you choose?
- A
128 tokens
- B
512 tokens
- C
1024 tokens
- D
2048 tokens
Show answer and explanation
Correct answer: B
Explanation
The selection of an embedding model's context length should balance the need to capture the full context of the source documents with computational efficiency. Since the source documents average 500 tokens, a context length of 512 tokens is sufficient to capture the relevant information while optimizing for lower computational costs. Longer context lengths (e.g., 1024 or 2048 tokens) are unnecessary in this scenario and would increase resource usage without added benefit.
- A. Incorrect.
128 tokens is too short to capture sufficient context from the source documents, which average 500 tokens. This would likely result in incomplete embeddings.
- B. Correct.
512 tokens is appropriate as it is sufficient to handle the average document length (500 tokens) while keeping computational costs lower compared to models with longer context lengths.
- C. Incorrect.
1024 tokens is unnecessarily long for the given document size and query length. It would increase computational costs without a proportional improvement in relevance.
- D. Incorrect.
2048 tokens is excessive for this scenario. It would significantly increase computational costs and is only beneficial for much longer documents or queries.