Databricks Generative AI Engineer Associate Question 117
Single answerYou are building a retrieval-augmented generation (RAG) system using a large language model (LLM) and a vector database for document retrieval. Your documents vary in length, ranging from short paragraphs to lengthy reports. After initial testing, you notice the model struggles to retrieve relevant context from long documents. Which chunking strategy would be most appropriate to address this issue?
- A
Split documents into fixed-size chunks of 500 tokens, without overlap.
- B
Split documents into fixed-size chunks of 1000 tokens, with a 200-token overlap.
- C
Dynamically chunk documents based on semantic boundaries like sentences or paragraphs.
- D
Use no chunking and rely on the vector database to handle the entire document.
Show answer and explanation
Correct answer: B
Explanation
When dealing with long documents, splitting them into fixed-size chunks with overlap (e.g., 1000 tokens with 200-token overlap) ensures that contextual information is preserved across chunk boundaries. This improves the retrieval quality without overwhelming the model's token limit. Other strategies either lose critical context, are inefficient, or risk exceeding the model's capacity.
- A. Incorrect.
Splitting into fixed-size chunks of 500 tokens without overlap may lead to the loss of important context at chunk boundaries, reducing retrieval accuracy for long documents.
- B. Correct.
Splitting into fixed-size chunks of 1000 tokens with a 200-token overlap helps preserve context across chunk boundaries, improving retrieval accuracy for lengthy documents.
- C. Incorrect.
Dynamically chunking based on semantic boundaries can work in some cases but may not result in consistent chunk sizes, which can make retrieval and model processing less efficient.
- D. Incorrect.
Using no chunking forces the system to process entire documents, which can lead to token limit issues with the LLM, particularly for lengthy reports.