SnowPro Specialty: Gen AI Question 52
Single answerFine-tuningA retail company uses a large language model in Snowflake to generate customer support replies. The model produces fluent text, but it often uses the wrong refund policy language and misses the company's required tone. The support team has already validated that the base model has sufficient general language capability. They now want to improve adherence to their domain-specific policies while keeping implementation manageable and evaluation repeatable inside Snowflake. Which approach is MOST appropriate?
- A
Create a high-quality training dataset of prompt-response pairs that reflect the company's refund policies and preferred tone, then fine-tune an appropriate model and evaluate it against a held-out test set.
- B
Increase the model temperature during inference so the model explores more possible responses and is more likely to match the company's refund policy wording.
- C
Replace the fine-tuning effort with a larger context window so the model can infer the correct policy language from longer prompts without any curated examples.
- D
Fine-tune the model on raw support ticket logs exactly as stored, including inconsistent answers from agents, and skip a separate evaluation dataset because production feedback will reveal quality issues.
Show answer and explanation
Correct answer: A
Explanation
This scenario points to fine-tuning because the model already demonstrates general language ability but needs to better reflect company-specific behavior: refund-policy wording and required tone. In practice, the most effective path is to assemble a clean, representative dataset of prompt-response examples that encode the desired behavior, then validate results on a separate held-out test set. This aligns with common generative AI best practices: use high-quality supervised examples, avoid noisy or contradictory training data, and evaluate systematically before release. By contrast, inference parameters such as temperature affect generation style and randomness, not domain learning. Similarly, simply expanding context is not the same as adapting the model itself. Candidates should recognize that fine-tuning is best suited for teaching repeatable domain-specific patterns, while careful data curation and evaluation are critical to achieving reliable outcomes.
- A. Correct.
Correct. Fine-tuning is the appropriate approach when the base model already has general capability but needs to learn organization-specific style, terminology, and response patterns. A curated prompt-response dataset aligned to the desired refund policy language and tone is essential. Using a held-out evaluation set is also a best practice so the team can measure whether the fine-tuned model actually improves adherence and quality before production deployment.
- B. Incorrect.
Incorrect. Temperature controls randomness and creativity during generation; it does not teach the model new domain-specific policies or make it reliably follow company-approved wording. A higher temperature usually increases variation, which can actually reduce consistency for policy-sensitive customer support replies.
- C. Incorrect.
Incorrect. A larger context window can help when relevant policy text is supplied at inference time, but it is not a substitute for fine-tuning when the goal is to consistently shape outputs toward a company's preferred tone and policy phrasing. It also does not eliminate the need for curated examples if the organization wants durable behavioral adaptation rather than relying only on prompt length.
- D. Incorrect.
Incorrect. Fine-tuning on uncurated raw logs is risky because the model may learn inconsistent, outdated, or noncompliant responses. Skipping a separate evaluation dataset is also poor practice because it prevents objective assessment of whether the model generalized correctly. Production feedback is useful, but it should not replace pre-deployment evaluation.