SnowPro Specialty: Gen AI Question 82
Select 3Considerations (for example, latency, availability)A retail company is building a customer support assistant in Snowflake using Cortex AI functions. During peak shopping periods, users report slow responses and occasional failures when generating answers. The engineering team wants to improve the end-user experience without redesigning the entire application. Which TWO actions are the most appropriate to address latency and availability concerns?
- A
Implement application-level retry logic with timeout handling and graceful fallback messaging when Cortex requests fail or take too long
- B
Increase the size of the virtual warehouse that stores the support transcripts, because Cortex model inference latency is primarily determined by warehouse size
- C
Design the user experience to handle variable response times, such as by using asynchronous processing or status updates for long-running generations
- D
Assume Cortex AI functions provide identical latency and availability characteristics across all models and regions, so no additional resilience planning is needed
- E
Precompute and cache frequently requested generated answers or summaries for common support topics to reduce repeated inference calls during peak load
Show answer and explanation
Correct answers: A, C, E
Explanation
The best answers are options 1, 3, and 5 because they address real-world latency and availability concerns through application design rather than assuming the underlying model service will always respond quickly. In Snowflake generative AI solutions, teams should plan for variable response times, transient failures, and peak-load behavior. Common best practices include implementing retries and timeouts, providing graceful degradation or fallback messaging, designing asynchronous user flows where appropriate, and caching repeated outputs to reduce unnecessary inference calls. By contrast, simply increasing warehouse size does not directly solve managed model inference latency, and it is unsafe to assume identical availability or latency across models and regions. These recommendations align with general Snowflake Cortex usage guidance and broader production AI reliability best practices: architect for resiliency, optimize for repeated workloads, and design user-facing experiences that tolerate non-deterministic response times.
- A. Correct.
Correct. For generative AI applications, latency and transient service issues must be handled at the application layer. Retry logic, sensible timeouts, and user-friendly fallback behavior improve resilience when requests are delayed or temporarily unavailable. This is a practical best practice for production AI workloads where model services can have variable response times.
- B. Incorrect.
Incorrect. Increasing warehouse size may help SQL query performance or data preparation steps, but it does not directly control the latency of managed model inference in Cortex AI functions. This option reflects a common misconception that all AI response time issues are solved by scaling Snowflake compute.
- C. Correct.
Correct. Generative AI workloads often have variable latency depending on prompt size, model choice, and service conditions. Designing the UX for asynchronous or long-running requests helps maintain a good user experience and reduces the impact of occasional slower responses. This is especially relevant during peak traffic periods.
- D. Incorrect.
Incorrect. Latency and availability can vary by model, workload, and service conditions. Production systems should not assume uniform behavior across models or deployment contexts. Resilience planning is still required, including retries, fallback behavior, and testing under expected load.
- E. Correct.
Correct. Caching or precomputing outputs for common, repeated requests is a practical way to reduce both latency and dependency on live inference during peak periods. For support assistants, many questions are repetitive, so serving cached summaries or approved responses can improve response times and reduce failure exposure.