Databricks Generative AI Engineer Associate Question 303
Select 2You are building a Retrieval-Augmented Generation (RAG) application on Databricks that uses a large language model (LLM) for answering user queries based on your enterprise's proprietary documents. To control costs associated with LLM usage, which of the following strategies can be implemented on Databricks?
- A
Use Databricks Feature Store to cache embeddings and retrieve them instead of calling the LLM repeatedly.
- B
Reduce the context size of prompts sent to the LLM by performing pre-filtering of retrieved documents.
- C
Employ a smaller, fine-tuned LLM hosted on Databricks instead of calling an external API for a larger base model.
- D
Increase the number of tokens sent to the LLM to improve response accuracy regardless of cost considerations.
- E
Utilize Databricks Auto Loader to batch queries and send them to the LLM in bulk to save costs.
Show answer and explanation
Correct answers: B, C
Explanation
To control costs in a RAG application, it is essential to minimize the number of tokens sent to the LLM and reduce dependency on expensive external APIs by using smaller, fine-tuned models. Pre-filtering retrieved documents reduces the prompt size, thereby lowering token usage, while hosting a fine-tuned model on Databricks eliminates external API costs.
- A. Incorrect.
Incorrect. While the Databricks Feature Store is useful for managing and sharing features in ML pipelines, it is not directly applicable for caching embeddings or reducing LLM costs in a RAG application.
- B. Correct.
Correct. Reducing the context size by pre-filtering retrieved documents before sending them to the LLM ensures that fewer tokens are processed, which directly reduces LLM usage costs.
- C. Correct.
Correct. Using a smaller, fine-tuned LLM hosted on Databricks can significantly reduce costs compared to relying on large external models through API calls.
- D. Incorrect.
Incorrect. Increasing the number of tokens sent to the LLM increases costs and is counterproductive to cost control.
- E. Incorrect.
Incorrect. Databricks Auto Loader is used for efficient ingestion of streaming data, and it cannot be used to batch queries for LLMs.