DEA-C01 Question 224
Select 3You are building a data pipeline on AWS to process and analyze real-time streaming data from IoT sensors. The data must be ingested with low latency, stored temporarily for processing, and then written to a data lake on Amazon S3. You also need to ensure that the solution is fault-tolerant and can scale automatically to handle fluctuating data volumes. Which combination of AWS services would best meet these requirements?
- A
Amazon Kinesis Data Streams to ingest the data, AWS Lambda to process the data, and Amazon S3 for storage
- B
Amazon SQS to ingest the data, AWS Glue to process the data, and Amazon RDS for storage
- C
Amazon Kinesis Data Firehose to ingest and process the data, and Amazon S3 for storage
- D
Amazon DynamoDB Streams to ingest the data, AWS Batch to process the data, and Amazon Aurora for storage
- E
Amazon MSK (Managed Streaming for Apache Kafka) to ingest the data, AWS Lambda to process the data, and Amazon S3 for storage
Show answer and explanation
Correct answers: A, C, E
Explanation
A successful pipeline for real-time streaming must handle low-latency ingestion, processing, and scalable storage. Amazon Kinesis Data Streams and Amazon MSK are both designed for real-time ingestion, AWS Lambda provides serverless and scalable processing, and Amazon S3 serves as a durable, scalable, and cost-effective storage solution for data lakes. Amazon Kinesis Data Firehose simplifies the pipeline by combining ingestion, processing, and direct integration with S3.
- A. Correct.
Correct: Amazon Kinesis Data Streams can handle real-time ingestion with low latency, AWS Lambda provides serverless and scalable data processing, and S3 is ideal for data lake storage.
- B. Incorrect.
Incorrect: Amazon SQS is designed for message queuing, not real-time streaming, and AWS Glue is better suited for batch ETL workflows. Amazon RDS is a relational database, which is not optimal for a data lake.
- C. Correct.
Correct: Amazon Kinesis Data Firehose can both ingest and process streaming data, and it integrates natively with Amazon S3 for storage with minimal configuration.
- D. Incorrect.
Incorrect: While DynamoDB Streams can capture changes in DynamoDB, it is not designed for high-throughput real-time ingestion. AWS Batch is not suitable for real-time processing, and Aurora is a relational database not suited for data lakes.
- E. Correct.
Correct: Amazon MSK provides a managed Apache Kafka service for real-time ingestion, AWS Lambda can process streaming data serverlessly, and S3 is a cost-effective storage option for data lakes.