Databricks Generative AI Engineer Associate Question 298
Select 3You have deployed a Retrieval-Augmented Generation (RAG) application in Databricks and are using inference logging to monitor its performance. You notice that the application occasionally returns irrelevant responses to user queries. Which of the following actions, based on insights from inference logs, can help improve the performance of your RAG application?
- A
Analyze the query embeddings and update the vector store with more diverse documents.
- B
Examine the latency metrics in the inference logs and increase the timeout threshold for external API calls.
- C
Identify low-confidence responses in the logs and fine-tune the language model on a domain-specific dataset.
- D
Use inference logs to filter out noisy queries and preprocess future inputs to improve input quality.
- E
Disable inference logging to reduce the overhead and improve application performance.
Show answer and explanation
Correct answers: A, C, D
Explanation
Inference logging provides valuable insights for diagnosing and improving the performance of a RAG application. By analyzing logs to identify issues such as irrelevant responses, low-confidence outputs, or noisy inputs, engineers can take targeted actions like updating the vector store, fine-tuning the model, or preprocessing inputs. These actions ensure that the application retrieves and generates more accurate and relevant responses.
- A. Correct.
Analyzing query embeddings and updating the vector store with more diverse documents can enhance the relevance of retrieved information, directly addressing the issue of irrelevant responses.
- B. Incorrect.
While examining latency metrics and adjusting timeout thresholds can improve response times, it does not address the issue of irrelevant responses in this scenario.
- C. Correct.
Identifying low-confidence responses and fine-tuning the language model on domain-specific data can help the model generate more accurate and relevant outputs.
- D. Correct.
Filtering noisy queries and preprocessing inputs based on inference logs can improve the quality of the input data, leading to better responses.
- E. Incorrect.
Disabling inference logging reduces monitoring capabilities and does not help improve the performance or relevance of the RAG application.