Databricks Generative AI Engineer Associate Question 52
Single answerYou are building a Retrieval-Augmented Generation (RAG) application to provide concise and accurate responses using a large corpus of documents as the source. During testing, you notice that the responses often include irrelevant information or are overly verbose. What is the most effective strategy to ensure extraneous content in the source documents does not degrade the quality of the application?
- A
Use document chunking to split documents into smaller segments for better retrieval.
- B
Apply a content filtering pipeline to remove irrelevant or low-quality sections from the source documents before indexing.
- C
Increase the embedding dimensionality of the vector database to capture more contextual information.
- D
Train a custom language model to identify and exclude irrelevant content during the inference stage.
Show answer and explanation
Correct answer: B
Explanation
Filtering extraneous content in the source documents before indexing is a critical step in improving the quality of a RAG application. By removing irrelevant or low-quality sections, the retriever and generator are provided with cleaner, more relevant data, which leads to more accurate and concise responses. While other options may address related challenges, they are not as effective or direct in solving the problem of extraneous content in the source documents.
- A. Incorrect.
While document chunking is useful for splitting large documents into smaller, retrievable pieces, it does not directly address the issue of filtering out extraneous content.
- B. Correct.
Using a content filtering pipeline ensures that irrelevant or low-quality sections are removed from the source documents before they are indexed, significantly improving the quality of the RAG application's outputs.
- C. Incorrect.
Increasing embedding dimensionality might capture more contextual information but does not specifically address filtering extraneous content in the source documents.
- D. Incorrect.
Training a custom language model to exclude irrelevant content during inference is complex and resource-intensive, and it is not the most direct or effective solution for this specific issue.