SnowPro Specialty: Gen AI Question 140
Select 2Considerations (e.g. capability, latency, and cost)A retail company is building a Snowflake-native assistant that answers product policy questions for customer support agents. During testing, the team finds that response quality is good with a large model, but average latency is too high for live chat and inference cost is exceeding budget. The team still needs acceptable answer quality for common policy questions and wants to optimize the solution before production. Which TWO actions are the most appropriate?
- A
Use a smaller model for the initial response path and reserve the larger model only for escalated or complex questions
- B
Increase the Snowflake virtual warehouse size because LLM inference latency is primarily determined by warehouse compute
- C
Reduce prompt and retrieved context length so the model processes fewer tokens per request
- D
Disable retrieval and send only the user question to the model, because removing context always improves both quality and cost
- E
Route every request to the most capable model to avoid quality variation, then control cost by shortening the final answer only
Show answer and explanation
Correct answers: A, C
Explanation
The best answer is to combine model routing with token optimization. In practical Snowflake GenAI solutions, teams should evaluate capability, latency, and cost together rather than optimizing only one dimension. Common best practices include selecting the smallest model that meets quality requirements for the majority of tasks, escalating only complex cases to more capable models, and minimizing prompt/context size so the model processes fewer tokens. These approaches are consistent with standard LLM design guidance: model choice and token volume are primary drivers of inference cost and latency, while retrieval should be preserved when grounded answers are required. Candidates should recognize that increasing warehouse size is not the main lever for external model inference performance, and removing retrieval entirely is risky for factual support use cases.
- A. Correct.
Correct. Using a smaller, lower-cost model for the common path is a practical optimization when most requests are routine. A larger model can be used selectively for harder cases that require deeper reasoning or higher-quality generation. This balances capability, latency, and cost through workload-based model routing rather than applying the most expensive option to every request.
- B. Incorrect.
Incorrect. Warehouse sizing affects Snowflake compute operations such as data processing, but model inference latency for LLM calls is not primarily solved by scaling the virtual warehouse. In a GenAI workflow, retrieval and data preparation may benefit from warehouse tuning, but the model's own inference characteristics, network round trips, and token volume are major latency drivers.
- C. Correct.
Correct. Token count strongly influences both latency and cost in LLM applications. Shorter prompts and tighter retrieval context reduce the amount of text the model must process, which commonly lowers inference time and usage cost. This is a standard optimization, provided the team preserves the minimum context needed for acceptable answer quality.
- D. Incorrect.
Incorrect. Removing retrieval context can lower token usage, but it does not reliably improve quality. For product policy questions, retrieval often provides the grounded, up-to-date information needed for accurate responses. Disabling retrieval may reduce cost but can increase hallucinations or incomplete answers, which is a poor tradeoff for support scenarios.
- E. Incorrect.
Incorrect. Sending all requests to the most capable model usually increases both latency and cost. Shortening only the final answer may help somewhat, but it does not address the cost and latency of input tokens, retrieved context, or unnecessary use of a premium model for simple requests. The misconception is treating model capability as the only meaningful control when routing and prompt optimization are often more effective.