Databricks Generative AI Engineer Associate Question 299
Select 3You are tasked with monitoring the performance of a Retrieval-Augmented Generation (RAG) application deployed on Databricks that generates responses to user queries using a combination of a vector store and a language model. You decide to enable inference logging to identify performance bottlenecks and improve the system. Which of the following metrics or data points can be captured using inference logging to assess the application's performance?
- A
Latency of individual components such as the vector store query and model inference
- B
Number of tokens used in each response generated by the language model
- C
User satisfaction scores collected via a feedback mechanism
- D
Errors or anomalies encountered during the RAG pipeline execution
- E
The financial cost of each query processed by the RAG application
Show answer and explanation
Correct answers: A, B, D
Explanation
Inference logging is a critical tool for monitoring the performance of a deployed RAG application. It helps capture objective metrics such as latency, token usage, and errors during execution. These metrics can be used to identify and address bottlenecks, improve efficiency, and enhance the overall performance of the system. However, subjective metrics like user satisfaction and derived metrics like financial cost are not directly captured by inference logging.
- A. Correct.
Correct: Latency of individual components such as the vector store query and model inference can be captured through inference logging to diagnose performance bottlenecks in the system.
- B. Correct.
Correct: The number of tokens used in each response is often logged during inference to monitor resource utilization and optimize cost.
- C. Incorrect.
Incorrect: User satisfaction scores are subjective metrics collected via external mechanisms and are not part of inference logging.
- D. Correct.
Correct: Errors or anomalies in the execution pipeline are typically captured in inference logs to identify and address issues in the application.
- E. Incorrect.
Incorrect: Financial cost is not directly logged through inference logging but can be inferred indirectly using other metrics like token usage and compute time.