Databricks Generative AI Engineer Associate Question 301
Select 4You are building a Retrieval-Augmented Generation (RAG) application on Databricks that uses a large language model (LLM) for answering user queries. The application frequently queries the LLM, leading to high costs. Which strategies can you implement using Databricks features to control these costs while maintaining application performance?
- A
Implement a caching mechanism using Databricks Delta tables to store frequently queried LLM responses.
- B
Use fine-tuning on a smaller, domain-specific LLM instead of relying on a general-purpose large LLM.
- C
Leverage Databricks AutoML to replace the LLM entirely for generating responses.
- D
Optimize the retrieval step by using vector databases like Databricks Lakehouse with embeddings to minimize unnecessary LLM calls.
- E
Batch user queries and process them together to reduce the number of individual LLM calls.
Show answer and explanation
Correct answers: A, B, D, E
Explanation
Controlling LLM costs in RAG applications involves reducing the frequency and scope of LLM queries while maintaining performance. Strategies like caching responses, fine-tuning smaller LLMs, optimizing retrieval steps, and batching queries are effective cost-saving techniques that can be implemented using Databricks features. Replacing the LLM entirely with AutoML is not a valid approach, as it does not fulfill the generative requirements of the RAG workflow.
- A. Correct.
Caching frequently queried responses in Databricks Delta tables reduces the number of repeated LLM queries, thereby lowering costs. It is an effective and scalable approach supported in Databricks.
- B. Correct.
Using a smaller, fine-tuned LLM tailored for your specific domain reduces costs while still delivering high-quality results, as smaller models are cheaper to run.
- C. Incorrect.
Databricks AutoML focuses on traditional machine learning tasks and is not designed to replace LLMs for natural language generation. This is not a suitable option for RAG applications.
- D. Correct.
Optimizing the retrieval step by using vector databases like Databricks Lakehouse ensures only the most relevant information is sent to the LLM, reducing unnecessary queries and associated costs.
- E. Correct.
Batching user queries allows you to process multiple inputs together, which can reduce the frequency of LLM calls and improve cost efficiency.