SnowPro Specialty: Gen AI Question 175
Single answerMulti-turn architectureA retail company is building a customer-support chatbot in Snowflake that uses Cortex AI to answer questions about orders, returns, and loyalty points. During testing, users report that follow-up questions such as "What about my last order?" or "Can you summarize that policy?" often lose context after several turns, leading to incorrect or generic answers. The engineering team wants a multi-turn architecture that preserves relevant conversational context while controlling token usage and reducing the chance of irrelevant prior messages affecting responses. Which approach is the BEST fit for this requirement?
- A
Send the entire raw conversation history with every prompt so the model can infer all prior context without any additional state management.
- B
Store conversation state externally, keep a rolling window of recent turns, and summarize older exchanges so only the most relevant context is included in subsequent prompts.
- C
Avoid maintaining conversation history and instead fine-tune a separate model for each customer so the model inherently remembers prior interactions.
- D
Concatenate only the latest user message with static system instructions, because large language models can reliably infer omitted earlier turns from the current question alone.
Show answer and explanation
Correct answer: B
Explanation
The key design principle in multi-turn architecture is that the application, not the model, is responsible for managing conversational state. In practice, a strong architecture stores prior turns outside the model, selectively includes recent messages, and summarizes or otherwise compresses older content to stay within token and cost constraints. This supports better relevance, lower latency, and more predictable behavior than repeatedly sending the full transcript. These practices align with general LLM application design guidance for conversational systems, including prompt-window management, conversation memory strategies, and context engineering patterns used with Snowflake Cortex AI applications.
- A. Incorrect.
This is not the best approach for a production multi-turn architecture. While sending full history may preserve context initially, it increases token consumption, latency, and cost as conversations grow. It also raises the risk that stale or irrelevant turns will influence the model's answer. A common best practice is to manage context explicitly rather than passing all prior messages indefinitely.
- B. Correct.
This is the best answer. In multi-turn conversational systems, the application typically manages conversation state outside the model, then supplies only the context needed for the next turn. Using a rolling window for recent turns plus summaries of older exchanges helps preserve continuity while controlling prompt size. This approach also reduces the chance that irrelevant earlier content will distort later responses.
- C. Incorrect.
This is incorrect because per-customer fine-tuning is not an appropriate or practical solution for routine conversational memory. Multi-turn context should generally be handled through application-managed state and prompt construction, not by expecting a model to permanently remember prior sessions for each user. This option reflects a misconception between model customization and runtime conversation management.
- D. Incorrect.
This is incorrect because large language models do not infer omitted conversation context reliably when it is not provided. If a user asks a referential follow-up like "that policy" or "my last order," the model needs explicit conversational state or retrieved context to answer accurately. Relying only on the latest message often causes context loss in multi-turn interactions.