SnowPro Specialty: Gen AI Question 271
Single answer3.4 Use Snowflake AI observability tools.A data science team deployed a text classification model in Snowflake using Model Serving and registered it in the Model Registry. After a recent upstream change to the source data, business users report that prediction quality has declined. The team wants to use Snowflake's AI observability capabilities to detect whether the production input data has drifted from the training data and to compare recent inference behavior with the model's baseline. Which action should they take?
- A
Use model monitoring in Snowflake to create a monitor for the deployed model, define a baseline from training/reference data, and track production metrics such as feature drift and prediction behavior over time.
- B
Query QUERY_HISTORY to compare warehouse execution times before and after the upstream change, because longer runtimes indicate model drift in production predictions.
- C
Rebuild the model immediately and overwrite the existing registry entry, because Snowflake observability tools do not support comparing production data to a baseline dataset.
- D
Use ACCESS_HISTORY to inspect which roles queried the model endpoint, because changes in role usage are the primary signal Snowflake uses to detect data drift.
Show answer and explanation
Correct answer: A
Explanation
The best answer is to use Snowflake's model monitoring and AI observability capabilities to compare production inference traffic against a baseline or reference dataset. In realistic ML operations, a drop in quality after an upstream data change often signals data drift or changes in prediction distributions. Snowflake's monitoring approach is intended for this exact use case: tracking model behavior over time and identifying when production inputs deviate from training expectations. By contrast, QUERY_HISTORY and ACCESS_HISTORY are valuable account observability tools, but they do not provide ML-specific drift detection. Snowflake documentation and best practices for MLOps emphasize using Model Registry together with model monitoring/observability to evaluate production health before retraining or redeployment decisions.
- A. Correct.
Correct. Snowflake AI observability for ML focuses on model monitoring, where you define a baseline or reference dataset and monitor production inputs and outputs for issues such as feature drift and changes in prediction distributions. In this scenario, the team needs observability into whether the incoming production data differs from training/reference data after the upstream pipeline change. Creating a model monitor is the appropriate action.
- B. Incorrect.
Incorrect. QUERY_HISTORY is useful for performance and operational troubleshooting, such as SQL execution duration, errors, and resource consumption. It does not provide ML-specific observability such as feature drift, prediction drift, or comparison of serving data against a model baseline. Someone might choose this because the issue appeared after a pipeline change, but runtime metadata is not the right tool for validating model quality degradation.
- C. Incorrect.
Incorrect. Rebuilding the model may eventually be necessary, but it should not be the first step. Snowflake provides model observability and monitoring capabilities specifically so teams can detect and diagnose issues like drift before deciding on retraining. This option reflects the misconception that model quality problems must be handled only by retraining rather than by first measuring and confirming the root cause.
- D. Incorrect.
Incorrect. ACCESS_HISTORY helps with governance, auditing, and understanding object access patterns. It is not designed to measure ML data drift or inference behavior. A candidate might pick this if they associate observability with auditability, but for AI observability the relevant capability is model monitoring, not access auditing.