Databricks Generative AI Engineer Associate Question 115
Single answerYou are designing a retrieval-augmented generation (RAG) pipeline using a large language model (LLM) for a customer support application. The documents being used are long technical manuals with dense information. During evaluation, you observe that the model struggles to retrieve relevant information when the chunks are too large but performs poorly at maintaining context when chunks are too small. Which chunking strategy would be most appropriate in this scenario?
- A
Split the documents into fixed-size chunks, such as 100 tokens, regardless of content boundaries.
- B
Use semantic chunking to split the documents based on logical sections or topics.
- C
Split the documents into overlapping chunks of a fixed size, such as 200 tokens, to preserve context.
- D
Avoid chunking entirely and process the entire document as a single input to the model.
Show answer and explanation
Correct answer: B
Explanation
Semantic chunking is the best approach for long, dense technical manuals because it maintains the logical and contextual integrity of the information. This helps the retrieval model deliver relevant and coherent information to the LLM, addressing the challenge of balancing relevance and context.
- A. Incorrect.
Splitting into fixed-size chunks of 100 tokens may result in incomplete or disjointed information, as it ignores the semantic structure of the content.
- B. Correct.
Semantic chunking ensures that chunks are meaningful and contextually complete by splitting based on logical sections or topics, which is ideal for dense technical manuals.
- C. Incorrect.
While overlapping chunks can help retain context, they may introduce redundancy and inefficiency, especially with dense documents. Semantic chunking is more effective in this scenario.
- D. Incorrect.
Processing the entire document as a single input is not feasible for long technical manuals due to token limits in most LLMs, and it can also lead to poor retrieval performance.