SnowPro Specialty: Gen AI Question 262
Single answerToken cost implicationsA retail analytics team is using Snowflake Cortex AI_COMPLETE to generate short product summaries for a customer-facing app. After a pilot, finance reports that inference spend is higher than expected. The team reviews the workload and finds that many prompts include long product descriptions, extensive formatting instructions repeated on every request, and a request for up to 800 output tokens even though the app displays only 2-3 sentences. The team wants to reduce token-related cost without changing the business outcome. Which action is the BEST recommendation?
- A
Reduce prompt and completion token usage by shortening the product context, removing repeated instructions where possible, and lowering the maximum output tokens to match the actual response length needed.
- B
Increase the model temperature so the model produces more creative answers in fewer tokens, which typically lowers token cost.
- C
Switch all requests to a larger, more capable model because larger models usually require fewer total tokens for the same task.
- D
Keep the current prompts unchanged and instead run the same generation requests more frequently in smaller batches, since token cost is primarily driven by batch size rather than prompt and response length.
Show answer and explanation
Correct answer: A
Explanation
For Snowflake Cortex generative AI workloads, a key cost driver is token consumption: both the tokens in the prompt and the tokens generated in the response. In real-world optimization, teams should minimize unnecessary context, avoid verbose repeated instructions, and constrain response length to what the application truly requires. This aligns with common best practices for LLM cost control: prompt efficiently, retrieve only the most relevant context, and set reasonable output limits. In this scenario, the strongest recommendation is to reduce prompt bloat and cap output tokens appropriately, because those changes directly address the source of excess token spend without changing the business outcome.
- A. Correct.
Correct. Token-related cost is driven largely by the number of input and output tokens processed. In this scenario, the team is sending unnecessarily long context, repeating instructions in every request, and allowing far more output tokens than the application actually needs. Trimming prompt content to only relevant information, reducing repetitive instructions where feasible, and lowering the maximum completion length are the most direct ways to reduce token consumption while preserving the required business result.
- B. Incorrect.
Incorrect. Temperature affects randomness and creativity, not token pricing directly. A higher temperature does not reliably reduce the number of input or output tokens. Candidates may choose this because generation settings can affect behavior, but token cost is primarily tied to how many tokens are sent to and generated by the model.
- C. Incorrect.
Incorrect. Larger models do not inherently reduce token usage or cost. In practice, they are often more expensive for inference. While a more capable model could sometimes improve quality, that does not make it the best answer for reducing token-related spend in this scenario, where the clearest issue is excessive prompt and output length.
- D. Incorrect.
Incorrect. Batch size is not the primary factor described here. If the same total text is processed, splitting requests into smaller batches does not solve the underlying problem of excessive prompt and completion tokens. This distractor reflects a common misunderstanding between operational throughput patterns and per-request token consumption.