SnowPro Advanced: Security Engineer Question 424
Single answerMonitor AI application performance metrics related to security and data qualityA financial services company has deployed an LLM-powered support assistant that uses Snowflake Cortex AISQL functions over curated customer-support data. The security engineering team must monitor the application for signs of data-quality drift and potential security issues, including exposure of sensitive fields in prompts or model outputs. They want an approach that provides measurable, reviewable evidence over time inside Snowflake. Which approach best meets these requirements?
- A
Persist prompt, response, user context, and evaluation results in Snowflake tables; then run scheduled evaluations for data quality and security-related checks, such as whether restricted data appears in outputs, and trend the results in dashboards/alerts.
- B
Rely on warehouse-level resource monitors because changes in credit consumption are the most reliable indicator of prompt leakage and declining response quality.
- C
Use masking policies on source tables only, and assume that if source columns are protected, there is no need to log prompts, responses, or evaluation metrics for the AI application.
- D
Monitor only QUERY_HISTORY for the AISQL statements because SQL text and execution status alone provide sufficient evidence of output quality, hallucination risk, and sensitive-data exposure.
Show answer and explanation
Correct answer: A
Explanation
For AI security engineering, the key distinction is between infrastructure monitoring and application-level monitoring. Snowflake operational metadata such as query history and warehouse metrics can show what ran and how much compute was consumed, but they do not by themselves evaluate whether an AI application is producing low-quality answers or exposing sensitive data. A stronger pattern is to persist AI interaction telemetry and evaluation outcomes in Snowflake, then measure trends over time. This supports governance, investigation, and alerting.
In Snowflake, best practice is to combine preventative controls with detective controls. Preventative controls include RBAC, masking policies, row access policies, and careful curation of data exposed to the model or retrieval layer. Detective controls include logging prompts/responses and running repeatable evaluations against security and quality criteria. This gives the security team measurable evidence for issues such as restricted data appearing in outputs, degraded response quality, or drift in retrieval context.
This aligns with general Snowflake security and governance practices: store auditable records in Snowflake, secure them with least privilege, and use scheduled SQL/tasks/dashboards to monitor trends and trigger alerts. The correct answer is therefore the option that captures application telemetry and evaluation metrics over time, rather than relying only on infrastructure consumption data or SQL execution history.
- A. Correct.
Correct. The most effective approach is to create an auditable telemetry pipeline in Snowflake that stores AI interaction data and evaluation outcomes, then measure trends over time. For security and data quality, the team needs more than operational health metrics; they need application-level evidence such as prompt/response traces, whether outputs contain restricted content, and whether quality metrics are degrading. In practice, this means persisting relevant interaction data, applying role-based access controls to that telemetry, and using scheduled evaluation jobs and dashboards/alerts to detect drift or security regressions.
- B. Incorrect.
Incorrect. Resource monitors help control and observe credit usage, but they do not measure whether sensitive information was exposed in model inputs or outputs, nor do they assess answer quality. A spike or drop in compute usage may indicate operational change, but it is not a reliable security or data-quality metric for an AI application.
- C. Incorrect.
Incorrect. Masking policies are important preventative controls for protecting sensitive data in tables, but they do not replace monitoring. Sensitive information can still appear in prompts, retrieved context, or generated outputs depending on application design and user behavior. Without logging and evaluating AI interactions, the team cannot produce reviewable evidence of whether the application is leaking data or degrading in quality over time.
- D. Incorrect.
Incorrect. QUERY_HISTORY can help with operational auditing of SQL execution, but it does not by itself provide semantic quality metrics or a direct measure of sensitive-data leakage in generated content. Execution metadata and SQL text are useful for troubleshooting and governance, but they are insufficient for evaluating hallucinations, output safety, or application-level data-quality drift.