DEA-C01 Question 85
Select 2You are a data engineer working for a company that processes large volumes of IoT sensor data in real-time. The data must be ingested, processed, and stored for analysis. The company wants to minimize operational overhead and ensure data durability, scalability, and near real-time processing. Which combination of AWS services should you use to design this solution?
- A
Amazon Kinesis Data Streams for data ingestion and Amazon Kinesis Data Analytics for processing
- B
Amazon S3 for real-time ingestion and Amazon Redshift for processing
- C
Amazon DynamoDB Streams for ingestion and AWS Glue for processing
- D
Amazon Kinesis Data Firehose for ingestion and Amazon S3 for storage
- E
Amazon Managed Streaming for Apache Kafka (MSK) for ingestion and AWS Lambda for processing
Show answer and explanation
Correct answers: A, D
Explanation
For real-time processing of IoT sensor data with minimal operational overhead, Amazon Kinesis Data Streams or Kinesis Data Firehose can be used for ingestion. Kinesis Data Streams enables real-time data ingestion with low latency, while Kinesis Data Firehose can ingest and deliver data to Amazon S3 or other destinations. Amazon Kinesis Data Analytics complements this by providing near real-time data processing capabilities. This combination ensures scalability, durability, and near real-time processing while reducing operational complexity.
- A. Correct.
Correct. Amazon Kinesis Data Streams can handle real-time ingestion of high-velocity data, and Amazon Kinesis Data Analytics can process data in near real-time with minimal operational overhead.
- B. Incorrect.
Incorrect. Amazon S3 is designed for durable storage, not real-time ingestion. While Amazon Redshift is a powerful analytics engine, it is not optimized for real-time processing.
- C. Incorrect.
Incorrect. While DynamoDB Streams can capture data changes in real-time, it is not suitable for high-volume ingestion from IoT devices. AWS Glue is more suitable for batch processing rather than real-time processing.
- D. Correct.
Correct. Amazon Kinesis Data Firehose is a fully managed service for real-time data ingestion to durable storage like Amazon S3, which is highly scalable and cost-effective for storing large amounts of data.
- E. Incorrect.
Incorrect. Amazon MSK is suitable for customized ingestion and processing pipelines but typically requires more operational overhead than Amazon Kinesis and may not fully align with the requirement to minimize operational overhead.