COF-C03 Question 228
Single answerSnowpipeA data engineering team needs to load JSON files into a Snowflake table within minutes of arrival. Files are written to an Amazon S3 bucket by an external application, and the team wants Snowflake to load only newly arrived files automatically without scheduling manual COPY commands. They created an external stage pointing to the S3 bucket and a target table, but new files are still not loading. Which action should the team take to enable continuous, event-driven loading with Snowpipe?
- A
Create a pipe with AUTO_INGEST=TRUE and configure S3 event notifications to send file arrival events to Snowflake
- B
Create a task that runs a COPY INTO command every minute against the external stage
- C
Resize the virtual warehouse used for loading so Snowpipe can detect new files faster
- D
Create a materialized view on top of the external stage so Snowflake can monitor new files automatically
Show answer and explanation
Correct answer: A
Explanation
The correct solution is to use Snowpipe with auto-ingest. In practice, this means creating a pipe that contains the COPY INTO logic and enabling AUTO_INGEST=TRUE, then configuring Amazon S3 event notifications so new file events are delivered to Snowflake. Snowpipe is designed for near-continuous loading of new files and uses Snowflake-managed compute rather than a customer-managed virtual warehouse. A scheduled task running COPY INTO can automate loads, but it is polling/scheduled orchestration rather than event-driven Snowpipe behavior. Best practice is to use auto-ingest when cloud event notifications are available and low-latency ingestion is required. See Snowflake documentation for Snowpipe, auto-ingest, and configuring Amazon S3 event notifications.
- A. Correct.
Correct. Snowpipe supports continuous data loading from files staged in cloud storage. For Amazon S3, event-driven auto-ingest requires a pipe defined with AUTO_INGEST=TRUE and cloud messaging/event notification configured so S3 sends notifications for new objects. This allows Snowflake to ingest newly arrived files automatically without repeatedly running COPY commands.
- B. Incorrect.
Incorrect. A task that executes COPY INTO on a schedule can automate loading, but it is not Snowpipe's event-driven auto-ingest model. The scenario specifically asks for continuous, event-driven loading within minutes of file arrival without scheduled manual COPY operations. Tasks are scheduled orchestration objects, not the mechanism Snowpipe uses for auto-ingest from S3 events.
- C. Incorrect.
Incorrect. Snowpipe uses Snowflake-managed compute, not a user-managed virtual warehouse, for file ingestion. Increasing warehouse size does not make Snowpipe detect files faster. This option reflects a common misconception that Snowpipe depends on a customer's warehouse in the same way bulk COPY operations do.
- D. Incorrect.
Incorrect. Materialized views cannot be created on an external stage to monitor file arrivals. External stages are storage references for files, not queryable objects in the way required for a materialized view. This distractor targets confusion between data ingestion objects and query optimization objects.