SnowPro Specialty: Gen AI Question 161
Single answerPerformance considerationsA retail company built a customer-support assistant in Snowflake using Cortex AISQL. The workflow stores support tickets in a table, chunks long tickets, embeds the chunks, and runs similarity search to retrieve context before sending the final prompt to a large language model for summarization and response generation. As ticket volume grows, users report slower response times and higher costs, especially during business hours when many analysts run the workflow concurrently. Which action would most directly improve performance for the retrieval step while also helping control overall latency in this RAG pipeline?
- A
Create and use a Cortex Search service over the chunked and embedded support content instead of repeatedly scanning and scoring the full chunks table at query time.
- B
Increase the max token setting for the final LLM call so the model can process more retrieved context in one request.
- C
Replace chunking with full-document embeddings so the retrieval query evaluates fewer rows.
- D
Run the entire workflow on a larger warehouse only for the final summarization step, leaving retrieval unchanged.
Show answer and explanation
Correct answer: A
Explanation
This question focuses on a practical performance issue in a Snowflake-based RAG pipeline: the retrieval stage often becomes the dominant source of latency as corpus size and concurrency increase. The best corrective action is to optimize retrieval itself rather than only the generation step. Snowflake best practices for Gen AI workloads emphasize efficient retrieval, limiting prompt size, and using purpose-built search capabilities where appropriate. Cortex Search is designed for low-latency search and retrieval use cases and is a better fit than repeatedly scanning chunk tables during each request. By contrast, increasing token limits usually raises latency and cost, and switching from chunked content to full-document embeddings generally harms retrieval relevance. A larger warehouse can help some query patterns, but if the architecture still performs inefficient runtime similarity work, the underlying bottleneck remains. In short, improving retrieval efficiency is the most direct way to improve user-perceived performance and control cost in this scenario.
- A. Correct.
Correct. In a RAG workflow, retrieval latency often comes from repeatedly comparing query embeddings against a large corpus at runtime. Using Cortex Search is designed to optimize low-latency search and retrieval over indexed content, which is more efficient than repeatedly scanning a table of chunks and computing similarity in an ad hoc way. This directly targets the retrieval bottleneck and can reduce end-to-end response time under concurrent load.
- B. Incorrect.
Incorrect. Increasing the max token setting usually increases model work, latency, and cost rather than improving retrieval performance. It may even worsen the overall user experience if the root cause is slow context retrieval. A larger context window is not a substitute for efficient search.
- C. Incorrect.
Incorrect. Replacing chunking with full-document embeddings is a common misconception. Larger documents typically reduce retrieval precision because relevant passages become diluted inside a single embedding. It may reduce row count, but it often hurts retrieval quality and can increase downstream prompt size if entire documents are passed to the model.
- D. Incorrect.
Incorrect. If the main slowdown is in retrieval, scaling only the final summarization step does not address the bottleneck. Warehouses can affect SQL execution performance, but leaving the expensive retrieval pattern unchanged means the pipeline still suffers from slow context lookup and unnecessary work.