SnowPro Specialty: Gen AI Question 138
Single answerChoosing a modelA retail company is building a Snowflake-native assistant that summarizes long customer support cases and drafts response suggestions for agents. The team plans to use Cortex AI functions and wants to minimize cost and latency while still producing reliable summaries for long, multi-step conversations. During testing, a smaller model responds quickly but often misses important details from lengthy case histories, while a larger model produces better results but increases response time and spend. Which approach is the BEST choice when selecting a model for this workload?
- A
Choose the largest available model for all requests because bigger models are generally the most accurate for every GenAI workload.
- B
Choose the smallest available model for all requests because lower latency is the primary factor in customer support scenarios.
- C
Evaluate candidate models against representative support conversations and select the smallest model that meets the application's quality requirements for summarization and drafting.
- D
Fine-tune the smaller model inside Snowflake Cortex so it can match the larger model's quality without additional evaluation.
Show answer and explanation
Correct answer: C
Explanation
When choosing a model in Snowflake GenAI scenarios, the recommended approach is to evaluate multiple candidate models against real application data, then balance quality, latency, and cost. For tasks like summarizing long support interactions and drafting agent responses, context handling and output quality matter, but overprovisioning with the largest model can unnecessarily increase spend and response times. A practical best practice is to use representative prompts, expected outputs, and measurable evaluation criteria to identify the smallest model that still meets business and quality requirements. This aligns with Snowflake guidance to select models based on workload characteristics and empirical testing rather than assumptions about model size alone.
- A. Incorrect.
Incorrect. Larger models often provide stronger reasoning and better handling of complex or long-context tasks, but they also typically increase cost and latency. Model selection should be driven by workload requirements and empirical evaluation, not by assuming the biggest model is always best.
- B. Incorrect.
Incorrect. Latency is important, especially in agent-assist scenarios, but choosing purely on speed can degrade output quality. In this scenario, the smaller model already showed that it misses important details in long support cases, so selecting it for all requests would not meet business requirements.
- C. Correct.
Correct. This reflects model selection best practice: test models on realistic data and choose the least expensive and lowest-latency option that still satisfies quality requirements. For long customer conversations, summarization accuracy and completeness are critical, so benchmark candidate models using representative prompts and outputs before choosing one.
- D. Incorrect.
Incorrect. This is a misconception. You should not assume fine-tuning is the default or necessary first step for model selection, and Snowflake Cortex does not make model choice irrelevant. The right first action is to evaluate available models for the target task. Even when customization is possible in broader GenAI workflows, model evaluation remains essential.