Databricks Generative AI Engineer Associate Question 291
Select 3You are building a Retrieval-Augmented Generation (RAG) application to answer customer queries using a large language model (LLM). To evaluate the performance of your RAG pipeline, you log results using MLflow. Which of the following actions are necessary to properly evaluate the model performance using MLflow?
- A
Log the model’s predictions and ground truth values as artifacts in MLflow.
- B
Track evaluation metrics like precision, recall, and F1 score as MLflow metrics.
- C
Log the vector database configuration details as an MLflow parameter.
- D
Log the latency of each RAG pipeline step (retrieval, generation, etc.) as an MLflow metric.
- E
Log the LLM weights and biases as an MLflow artifact.
Show answer and explanation
Correct answers: A, B, D
Explanation
To effectively evaluate the performance of a RAG application using MLflow, it is essential to log key metrics and artifacts that provide insight into the model's predictions, accuracy, and system efficiency. Logging predictions, ground truth values, evaluation metrics (precision, recall, F1 score), and pipeline step latencies ensures a comprehensive understanding of both the model's effectiveness and the system's performance.
- A. Correct.
Logging predictions and ground truth values as artifacts in MLflow is crucial for qualitative and quantitative evaluation of the model's performance.
- B. Correct.
Tracking evaluation metrics like precision, recall, and F1 score helps quantify the model’s effectiveness in generating accurate and relevant answers.
- C. Incorrect.
While the vector database configuration is important for the RAG pipeline, it is not directly related to evaluating the model's performance.
- D. Correct.
Logging the latency of RAG pipeline steps as MLflow metrics is important for understanding system performance and identifying bottlenecks, which indirectly impacts the evaluation of the application.
- E. Incorrect.
Logging LLM weights and biases is not required for evaluating the RAG model’s performance. This information is more relevant for model training and debugging.