SnowPro Specialty: Gen AI Question 277
Single answerComparisonsA retail company is evaluating two prompt-and-model combinations in Snowflake Cortex for generating product descriptions. The team has a table containing the source product attributes and two generated outputs per row: one from Variant A and one from Variant B. They want a scalable way to compare the quality of the two generated responses across thousands of rows using an LLM-as-a-judge approach, and they need the result to identify which variant performed better for each record. Which Snowflake capability is the most appropriate choice?
- A
Use AI_COMPLETE on each variant separately and compare token counts to determine which response is better
- B
Use a Cortex Analyst semantic model so business users can ask which variant is better in natural language
- C
Use the COMPARE function to evaluate the two generated outputs against each other for each row
- D
Use EMBED_TEXT_768 and select the response with the larger embedding vector magnitude
Show answer and explanation
Correct answer: C
Explanation
The best choice is to use COMPARE because the scenario is explicitly about comparing two generated outputs for the same input across many rows. In Snowflake Cortex evaluation workflows, comparison-based evaluation is appropriate when you want an LLM judge to determine which candidate output is better rather than scoring a single output in isolation. By contrast, AI_COMPLETE is for generation and can be used in custom evaluation patterns, but it is not the purpose-built comparative function here. Cortex Analyst addresses conversational analytics over data, not generated-text quality comparison. Embeddings support semantic search and similarity use cases, not direct quality judgment. This aligns with Snowflake best practices for using built-in evaluation capabilities when benchmarking prompt or model variants at scale.
- A. Incorrect.
Incorrect. AI_COMPLETE can generate or evaluate text depending on prompting, but simply comparing token counts does not measure response quality. Longer or shorter answers are not inherently better. This option reflects a common misconception that usage metrics can substitute for structured model-based evaluation.
- B. Incorrect.
Incorrect. Cortex Analyst is designed for natural language question answering over structured data using semantic models, not for row-by-row evaluation of generated text variants. While an analyst might use Cortex Analyst to explore aggregate results after evaluation, it is not the primary capability for performing side-by-side output comparison at scale.
- C. Correct.
Correct. The COMPARE function is intended for comparative evaluation scenarios where two candidate responses are judged against each other. In this scenario, the team already has Variant A and Variant B outputs and needs a scalable LLM-as-a-judge mechanism that identifies which response is better per record. This is exactly the type of applied evaluation workflow Comparisons supports.
- D. Incorrect.
Incorrect. EMBED_TEXT_768 produces embeddings for semantic similarity and retrieval-style use cases. The magnitude of an embedding vector does not indicate response quality. This distractor targets a common misunderstanding that embeddings can directly rank answer quality without a defined evaluation method.