AIF-C01 Question 61
Single answerYou are tasked with developing a text summarization application using a transformer-based large language model (LLM). During testing, you notice that the model generates incomplete sentences when summarizing long input documents. What is the best approach to address this issue?
- A
Use chunking to split the input document into smaller, manageable segments before feeding them to the model.
- B
Increase the model's token limit by adding more GPUs to your infrastructure.
- C
Reduce the output length of the model by setting a lower maximum token limit in the prompt.
- D
Switch to a multi-modal model to process the text content more effectively.
Show answer and explanation
Correct answer: A
Explanation
Transformer-based LLMs have token limits, and processing inputs that exceed these limits can lead to incomplete or truncated outputs. Chunking is a foundational technique in generative AI to divide lengthy inputs into smaller parts, allowing the model to process each part effectively without encountering token limitations. This ensures accurate and complete summaries for long documents.
- A. Correct.
Chunking is an effective method for handling lengthy inputs by dividing them into smaller, manageable segments, ensuring the model processes them without exceeding token limits. This directly addresses the problem of incomplete outputs.
- B. Incorrect.
The token limit of a model is determined by its architecture and cannot be arbitrarily increased by adding more GPUs. This option does not address the core issue.
- C. Incorrect.
Reducing the output length by setting a lower maximum token limit may further truncate the summary, worsening the issue rather than solving it.
- D. Incorrect.
Multi-modal models are designed to process multiple data types, such as text and images. Switching to a multi-modal model does not address the specific issue of handling long text inputs in a summarization task.