Databricks Generative AI Engineer Associate Question 118
Single answerYou are building a retrieval-augmented generation (RAG) system using a large language model (LLM) for a knowledge base containing lengthy technical documents. During development, you evaluate retrieval effectiveness and find that the model struggles to generate relevant answers when the input context is too long. Which chunking strategy should you select to optimize the retrieval and generation process?
- A
Split the documents into fixed-size chunks of a few hundred tokens, ensuring each chunk fits within the model's context window.
- B
Use overlapping sliding windows to create chunks, allowing some redundancy between consecutive chunks.
- C
Split the documents into large chunks to minimize the number of retrieval calls during inference.
- D
Do not chunk the documents and instead input entire documents into the model to preserve context.
Show answer and explanation
Correct answer: B
Explanation
Using overlapping sliding windows is a best practice in retrieval-augmented generation systems for lengthy documents. This strategy ensures that important context is not lost between chunks while keeping each chunk within the model’s token limit. It provides a balance between retrieval accuracy and the constraints of the LLM’s context window, leading to better retrieval and generation outcomes.
- A. Incorrect.
While fixed-size chunks can fit within the model’s context window, they may lose critical context when splitting arbitrarily, leading to less effective retrieval.
- B. Correct.
Overlapping sliding windows ensure that context is preserved between chunks, reducing the chance of losing important information during both retrieval and generation. This approach balances retrieval effectiveness and model constraints.
- C. Incorrect.
Large chunks may exceed the model’s context window, causing truncation and potentially losing critical information, which can degrade both retrieval and generation performance.
- D. Incorrect.
Inputting entire documents into the model is impractical for lengthy documents, as it likely exceeds the model’s context window and leads to ineffective generation.