SnowPro Specialty: Gen AI Question 81
Single answerConsiderations (for example, latency, availability)A retail company built a customer-support assistant in Snowflake using Cortex AISQL functions to summarize case history and draft suggested responses for agents. During peak holiday traffic, agents report that response generation is sometimes too slow for live conversations, and operations notes occasional temporary unavailability from the underlying model provider. The solution architect must improve the user experience without rewriting the application outside Snowflake. Which approach is the BEST recommendation?
- A
Implement a fallback strategy that uses a smaller or alternate supported model for interactive requests, and reserve larger models for non-interactive batch workloads
- B
Increase the size of the Snowflake warehouse that runs the SQL statements, because model inference latency is primarily determined by warehouse compute size
- C
Run the same prompt multiple times in parallel against the same model and return the first result, because this guarantees higher availability and lower cost
- D
Convert all interactive requests into scheduled batch jobs so that the application no longer depends on model response time during agent conversations
Show answer and explanation
Correct answer: A
Explanation
The best answer is to design for workload-aware model selection and graceful degradation. In Snowflake Cortex, different models have different latency, quality, and cost characteristics. For interactive applications, lower-latency models are often more appropriate, while larger models can be reserved for offline or asynchronous tasks. In addition, because availability can vary across underlying model providers, a fallback strategy helps preserve the user experience when a preferred model is unavailable or slower than expected. Snowflake documentation and best practices emphasize choosing models according to task requirements and operational considerations such as latency, throughput, availability, and cost. By contrast, simply increasing warehouse size does not directly address provider/model inference latency, duplicating requests wastes cost without guaranteeing resilience, and converting an interactive workflow to batch fails the business requirement.
- A. Correct.
Correct. For real-time agent assistance, latency and availability are key design considerations. A practical pattern is to choose models based on workload requirements: use lower-latency models for interactive experiences and larger, potentially slower models for offline or less time-sensitive tasks. A fallback to another supported model can also help maintain service continuity when a preferred model is temporarily unavailable. This aligns with Snowflake best practices for selecting models based on performance, cost, and use-case requirements rather than assuming one model fits every scenario.
- B. Incorrect.
Incorrect. Increasing warehouse size can help SQL execution performance around the request, but it does not make the external model itself inherently respond faster. For Cortex AISQL inference workloads, end-to-end latency is influenced significantly by model choice and provider-side behavior, not just warehouse sizing. This option reflects a common misconception that all latency in an AI workflow is solved by more database compute.
- C. Incorrect.
Incorrect. Sending duplicate inference requests in parallel to the same model generally increases cost and can add unnecessary load. It also does not guarantee improved availability, especially if the issue is with the model endpoint or provider itself. This is a plausible but flawed reliability pattern because it treats provider-side unavailability as if it were just a single-request performance problem.
- D. Incorrect.
Incorrect. Batch processing may be appropriate for back-office enrichment, summarization at scale, or offline analysis, but it does not solve the business requirement for live agent conversations. This option removes the interactive user experience instead of optimizing it. It confuses workload redesign with meeting the stated latency and availability goals.