SnowPro Specialty: Gen AI Question 275
Single answerEvaluation metricsA retail company uses a Snowflake-based GenAI application to generate customer-facing answers from an internal product knowledge base. During testing, the team notices that many responses sound fluent but occasionally include product details that are not supported by the retrieved documents. The team wants an evaluation approach that helps them compare prompt and retrieval changes while specifically reducing unsupported claims. Which metric should they prioritize as the primary success criterion?
- A
Groundedness, because it measures whether the generated answer is supported by the provided context
- B
Latency, because faster responses reduce the chance that the model invents information
- C
Token count, because shorter answers are less likely to contain unsupported claims
- D
Answer similarity to a reference response, because matching the expected wording is the best indicator that the answer is supported
Show answer and explanation
Correct answer: A
Explanation
For GenAI applications that answer questions using retrieved enterprise content, a key evaluation concern is whether the model's output is anchored in the source material. When users report fluent but unsupported responses, groundedness is the most appropriate primary metric because it directly assesses whether the generated answer is supported by the provided context. This aligns with common best practices for evaluating retrieval-augmented generation systems: use factuality and context-support metrics to detect hallucinations, while treating operational metrics such as latency and token usage as secondary optimization criteria. Reference-answer similarity can still be helpful in some benchmark-style evaluations, but it is usually less robust for real-world enterprise Q&A where multiple correct phrasings may exist. In practice, teams often track groundedness together with related metrics such as relevance and completeness, but groundedness should be prioritized when the explicit goal is to reduce unsupported claims.
- A. Correct.
Correct. Groundedness is the most relevant metric when the main problem is hallucination or unsupported statements in a retrieval-augmented workflow. It evaluates whether the response is justified by the retrieved context, making it the best primary metric for comparing prompt and retrieval changes when the goal is to reduce unsupported claims.
- B. Incorrect.
Incorrect. Latency is an operational performance metric, not a factual quality metric. While response time matters for user experience and system design, reducing latency does not directly measure or solve whether an answer is supported by retrieved evidence.
- C. Incorrect.
Incorrect. Token count can affect cost and readability, but it is not a reliable indicator of factual support. A short answer can still be hallucinated, and a longer answer can still be fully grounded. Using length as the primary quality metric would miss the actual issue.
- D. Incorrect.
Incorrect. Similarity to a reference answer can be useful in narrow tasks with well-defined expected outputs, but in open-ended enterprise Q&A there may be multiple valid answers. A response could look similar to a reference yet still include unsupported details, so this metric is less direct than groundedness for the stated problem.