Databricks Generative AI Engineer Associate Question 44
Select 3You are tasked with fine-tuning a language model to answer questions about a collection of lengthy technical documents. The model has a maximum token limit of 4096 tokens per input and output combined. Some of the documents contain very long sections that exceed this token limit. Which strategies should you apply to ensure the model processes the documents efficiently while respecting the token limit?
- A
Split the documents into smaller, meaningful chunks based on their structure, such as sections or paragraphs.
- B
Truncate the documents to fit within the token limit by discarding content from the end of the document.
- C
Use overlapping chunks to retain context between consecutive chunks of the document.
- D
Ignore the token limit and pass the full document to the model, as the model will automatically truncate the input.
- E
Summarize each document into a shorter version before passing it to the model.
Show answer and explanation
Correct answers: A, C, E
Explanation
When dealing with long documents that exceed the token limit of a model, chunking the document into smaller, meaningful pieces based on its structure (e.g., sections or paragraphs) allows the model to process each part efficiently. Overlapping chunks ensure context is preserved across chunks, and summarization is a useful technique when the document needs to be condensed further. Truncation and ignoring the token limit, however, can lead to loss of critical information or processing errors.
- A. Correct.
Splitting the documents into smaller chunks based on structure is an effective strategy to ensure each chunk fits within the token limit while maintaining logical context.
- B. Incorrect.
Truncating documents by discarding content risks losing critical information and is generally not a recommended approach for processing long documents.
- C. Correct.
Using overlapping chunks helps retain context and continuity between chunks, which is essential for models to provide coherent answers across sections.
- D. Incorrect.
Ignoring the token limit is not a viable strategy, as the model will not process tokens beyond its maximum limit, leading to incomplete or erroneous outputs.
- E. Correct.
Summarizing documents into shorter versions can help condense the information, making it possible to stay within the token limit while still conveying key points.