SnowPro Specialty: Gen AI Question 261
Single answerToken cost implicationsA data engineering team is building a customer-support assistant in Snowflake that uses Cortex AI complete functions to summarize case history and draft agent responses. During testing, finance reports that inference costs are much higher than expected. Review shows that each request sends the full case history, long system instructions, and several retrieved knowledge-base passages, even when only a short answer is needed. The team wants to reduce token-related cost without significantly hurting answer quality. Which action should they take FIRST?
- A
Reduce the number of input tokens sent with each request by shortening prompts, trimming unnecessary case history, and limiting retrieved context to the most relevant passages
- B
Increase the warehouse size so the model can process prompts more efficiently and reduce token billing
- C
Convert all prompts to uppercase because tokenizers typically produce fewer tokens for capital letters
- D
Raise the model temperature so the model generates shorter outputs and lowers total token usage
Show answer and explanation
Correct answer: A
Explanation
For Snowflake Cortex AI workloads, token cost is influenced by how many input and output tokens are processed by the selected model. In real-world applications, excessive prompt context is one of the most common causes of unnecessary cost. Best practices include keeping instructions concise, limiting retrieved context to the most relevant chunks, and avoiding repeated submission of long histories when a summary or shorter context window will suffice. Output length controls can also help, but the scenario indicates the biggest issue is oversized input payloads, so reducing input tokens is the best first action. This is consistent with general Snowflake Cortex usage guidance and LLM cost-optimization practices: optimize prompt size, retrieval relevance, and context management before looking for less direct changes such as infrastructure scaling or parameter tuning.
- A. Correct.
Correct. Token-related inference cost is primarily driven by the number of input and output tokens processed by the model. If the application is sending full conversation history, verbose instructions, and excessive retrieved context on every request, the most direct and reliable first optimization is to reduce unnecessary tokens. In practice, this means summarizing or truncating prior history, keeping system prompts concise, and retrieving only the most relevant passages instead of overloading the prompt. This aligns with prompt-engineering and retrieval best practices for controlling cost while preserving quality.
- B. Incorrect.
Incorrect. Warehouse size affects Snowflake compute used for SQL and data processing tasks, but it does not reduce the number of model tokens billed for Cortex AI inference. A larger warehouse may improve throughput for surrounding pipelines, but token cost is tied to model usage, not warehouse sizing. This option reflects a common misconception that all AI cost in Snowflake is warehouse-driven.
- C. Incorrect.
Incorrect. There is no general best practice or documented behavior showing that converting prompts to uppercase reduces token count. Tokenization depends on the model's tokenizer and text patterns, not a simple capitalization rule. In many cases, changing case could even make prompts less readable or alter model behavior without providing meaningful savings.
- D. Incorrect.
Incorrect. Temperature controls randomness and creativity in generation, not token pricing directly. While output length can affect total token usage, raising temperature does not reliably make responses shorter and may actually produce more variable or verbose outputs. If the goal is cost reduction, the better first step is to control prompt size and context volume explicitly.