ARA-C01 Question 303
Single answerKafkaA retail company streams order events from Apache Kafka into Snowflake for near-real-time analytics. The architecture team wants to minimize custom code, preserve Kafka record metadata for troubleshooting, and ensure offsets are committed only after records are durably written to Snowflake. Which approach best meets these requirements?
- A
Use the Snowflake Kafka Connector to write records into a Snowpipe Streaming channel and enable metadata columns so Kafka topic, partition, and offset can be stored with the ingested rows.
- B
Configure Kafka Connect to land Avro files in an internal Snowflake stage, then use a scheduled task with COPY INTO to load them and commit Kafka offsets immediately after the files are produced.
- C
Use a custom Kafka consumer application that inserts rows with JDBC into Snowflake tables and manually stores offsets in a separate control table after each poll cycle.
- D
Use Snowpipe auto-ingest directly from Kafka topics and configure event notifications so Snowflake can subscribe to Kafka offsets.
Show answer and explanation
Correct answer: A
Explanation
For Kafka-to-Snowflake ingestion, the recommended production pattern is to use the Snowflake Kafka Connector rather than build a custom consumer. This reduces operational complexity and aligns with Snowflake best practices for managed ingestion. In modern architectures, the connector can use Snowpipe Streaming to achieve low-latency ingestion without staging files in cloud object storage. The connector is also designed to coordinate ingestion state with Kafka offset commits so that offsets are committed only after records are successfully processed for Snowflake, which helps avoid data loss scenarios. Additionally, preserving Kafka metadata such as topic, partition, and offset is a common design requirement for observability and replay troubleshooting. Snowflake documentation for the Kafka Connector and Snowpipe Streaming describes these capabilities and the recommended architecture for near-real-time ingestion from Kafka.
- A. Correct.
Correct. The Snowflake Kafka Connector is the standard integration for streaming Kafka data into Snowflake with minimal custom development. It is designed to coordinate offset management with ingestion so offsets are committed only after data has been successfully processed for Snowflake. When used with Snowpipe Streaming, it supports low-latency ingestion without requiring files in cloud storage. The connector can also expose Kafka metadata such as topic, partition, and offset, which is useful for lineage, replay analysis, and troubleshooting duplicate or missing records.
- B. Incorrect.
Incorrect. Kafka Connect does not natively land files in an internal Snowflake stage as a standard Kafka-to-Snowflake pattern, and this approach adds unnecessary batch latency and orchestration complexity. More importantly, committing offsets immediately after file production does not guarantee the data has been durably loaded into Snowflake tables. This breaks the requirement that offsets should only be committed after durable write completion.
- C. Incorrect.
Incorrect. A custom JDBC consumer could potentially be built, but it does not minimize custom code and creates operational risk around error handling, retries, idempotency, and offset coordination. This option may appeal to teams wanting full control, but it is not the best architectural choice compared with the supported Snowflake Kafka Connector, which is purpose-built for this integration.
- D. Incorrect.
Incorrect. Snowpipe auto-ingest works with files arriving in supported cloud storage through event notifications, not directly from Kafka topics. Snowflake does not subscribe directly to Kafka offsets through Snowpipe auto-ingest. This option reflects a common misconception that Snowpipe can ingest from any event source, when in practice Kafka requires the Snowflake Kafka Connector.