Databricks Generative AI Engineer Associate Question 79
Select 3You are working on a Generative AI application using a retrieval-augmented generation (RAG) approach. To evaluate the performance of your retrieval system, you decide to use specific metrics. Which of the following metrics are appropriate to measure the retrieval performance of your system?
- A
Precision at K (P@K)
- B
Recall at K (R@K)
- C
BLEU score
- D
Mean Average Precision (MAP)
- E
Perplexity
Show answer and explanation
Correct answers: A, B, D
Explanation
To evaluate retrieval performance, metrics like Precision at K, Recall at K, and Mean Average Precision are used because they focus on the relevance and ranking of retrieved documents. BLEU and Perplexity are unrelated to retrieval systems, as they measure text generation quality and model uncertainty, respectively.
- A. Correct.
Precision at K (P@K) is a common metric for retrieval systems that measures the proportion of relevant documents among the top K retrieved documents. It is directly related to retrieval performance.
- B. Correct.
Recall at K (R@K) measures the proportion of relevant documents retrieved among all relevant documents available in the dataset. It is another important metric for evaluating retrieval systems.
- C. Incorrect.
BLEU score is a metric for evaluating the quality of text generation, not the performance of a retrieval system. It measures the similarity between generated text and reference text.
- D. Correct.
Mean Average Precision (MAP) is a standard metric for retrieval systems that evaluates the average precision across multiple queries, considering the rank of relevant documents.
- E. Incorrect.
Perplexity is a metric for evaluating language models and measures the uncertainty of predictions. It is not used to evaluate retrieval performance.