Databricks Generative AI Engineer Associate Question 119
Single answerYou are designing a question-answering system using a transformer-based language model. The system retrieves relevant documents from a large knowledge base and answers queries based on these documents. During testing, you notice the model struggles to extract meaningful information from long documents and often retrieves irrelevant information. Based on this observation, which chunking strategy should you select to optimize both retrieval and model performance?
- A
Divide documents into overlapping chunks of fixed size, such as 200-300 tokens.
- B
Split documents into chunks based on semantic boundaries, such as paragraphs or sections.
- C
Avoid chunking and process the entire document as a single input to the model.
- D
Use non-overlapping fixed-size chunks of 100 tokens to maximize retrieval speed.
Show answer and explanation
Correct answer: A
Explanation
When designing a retrieval-augmented question-answering system, the chunking strategy is crucial for balancing retrieval accuracy and model performance. Overlapping fixed-size chunks (200-300 tokens) ensure sufficient context is preserved while maintaining a manageable input size for the transformer model. This approach is particularly effective for long documents, where context at chunk boundaries is critical for generating accurate answers.
- A. Correct.
Dividing documents into overlapping chunks of fixed size ensures that context is preserved across chunk boundaries while maintaining a manageable input size for the model. This approach balances retrieval performance and model accuracy.
- B. Incorrect.
Splitting based on semantic boundaries can improve interpretability but may lead to uneven chunk sizes, potentially causing issues for models that require consistent input lengths.
- C. Incorrect.
Processing the entire document as a single input can exceed the model's token limit, leading to truncation and loss of critical information.
- D. Incorrect.
Using non-overlapping fixed-size chunks sacrifices context at the boundaries, which can negatively impact the model's ability to understand the full context of the query.