Databricks Generative AI Engineer Associate Question 47
Single answerYou are tasked with fine-tuning a large language model (LLM) to answer questions about a lengthy legal document. The document contains various sections, including titles, subsections, and detailed clauses. Given the model's token limit of 2048 tokens, which chunking strategy would be most appropriate to ensure the model receives coherent and contextually relevant input?
- A
Chunk the document by splitting it into equal-sized parts, ensuring each chunk is below the token limit.
- B
Chunk the document by splitting it at logical boundaries, such as sections or subsections, while keeping each chunk below the token limit.
- C
Split the document into overlapping chunks of text, regardless of the document's structure, to maximize context retention.
- D
Compress the document by removing all non-essential sections, then split it into chunks under the token limit.
Show answer and explanation
Correct answer: B
Explanation
For lengthy documents, such as legal texts, chunking by logical boundaries (e.g., sections, subsections) ensures that each input to the model is coherent and contextually meaningful. This approach aligns with the model's token constraints while preserving the document's structure, enabling more accurate generation of responses.
- A. Incorrect.
Splitting the document into equal-sized parts may lead to loss of context or breaking logical boundaries, making it harder for the model to interpret the input.
- B. Correct.
Chunking by logical boundaries, such as sections or subsections, ensures that each chunk is coherent and contextually relevant, while adhering to the token limit.
- C. Incorrect.
Using overlapping chunks can increase context retention, but it ignores the document's structure and may introduce redundancy, which is not ideal for structured documents like legal texts.
- D. Incorrect.
Removing non-essential sections could lead to information loss and may affect the quality of responses, especially for a detailed document like a legal text.