Databricks Generative AI Engineer Associate Question 46
Single answerYou are tasked with fine-tuning a generative AI model on a large document with a hierarchical structure, such as chapters, sections, and subsections. The model has a maximum token limit of 2048 tokens for each input. Which chunking strategy should you apply to ensure the model can process the document effectively without losing important context?
- A
Split the document into fixed-sized chunks of 2048 tokens, regardless of logical boundaries.
- B
Chunk the document by preserving logical boundaries such as chapters, sections, or subsections, and ensure each chunk stays within the model's token limit.
- C
Combine multiple chapters into a single chunk to maximize the token utilization.
- D
Use overlapping sliding windows across the document where each chunk overlaps by 50% of the token limit.
Show answer and explanation
Correct answer: B
Explanation
Chunking a document for generative AI models involves balancing token constraints with context preservation. Logical boundaries like chapters and sections provide natural divisions in the document, ensuring meaningful context is retained in each chunk. Splitting by fixed sizes or overlapping windows can disrupt this structure or introduce inefficiencies. Therefore, preserving logical boundaries within token limits is the best approach.
- A. Incorrect.
Splitting into fixed-sized chunks of 2048 tokens ignores the document's logical structure, which may lead to loss of contextual meaning or incomplete information within a chunk.
- B. Correct.
Preserving logical boundaries like chapters, sections, or subsections ensures that each chunk maintains meaningful context while adhering to the token limit, making it the optimal strategy.
- C. Incorrect.
Combining multiple chapters into a single chunk can exceed the token limit and disrupt processing, making this strategy impractical for large documents.
- D. Incorrect.
Using overlapping sliding windows may help with context preservation but can create redundancy and inefficiency, especially when the document has a clear hierarchical structure.