DEA-C01 Question 294
Select 3You are tasked with building a data processing pipeline that ingests data from IoT sensors, processes the data in near real-time, and stores the results in an Amazon S3 bucket for further analysis. The pipeline should be fully automated and scalable. Which combination of AWS services would best meet this requirement?
- A
Amazon Kinesis Data Streams for ingestion, AWS Lambda for processing, Amazon S3 for storage
- B
Amazon SQS for ingestion, AWS Glue for processing, Amazon RDS for storage
- C
AWS IoT Core for ingestion, AWS Lambda for processing, Amazon S3 for storage
- D
Amazon Kinesis Data Firehose for ingestion and processing, Amazon S3 for storage
- E
Amazon EMR for ingestion, processing, and storage
Show answer and explanation
Correct answers: A, C, D
Explanation
To automate data processing for IoT sensors in near real-time, you can use services like Amazon Kinesis (Data Streams or Data Firehose) or AWS IoT Core for ingestion, AWS Lambda for serverless processing, and Amazon S3 for scalable, cost-effective storage. This combination ensures scalability, real-time processing, and automation while adhering to best practices for data engineering on AWS.
- A. Correct.
Correct: Amazon Kinesis Data Streams is highly scalable for real-time ingestion, AWS Lambda automates processing, and Amazon S3 is a cost-effective storage solution.
- B. Incorrect.
Incorrect: Amazon SQS is not ideal for real-time ingestion, and Amazon RDS is not suited for large-scale data storage in this scenario.
- C. Correct.
Correct: AWS IoT Core is specifically designed for ingesting IoT sensor data, AWS Lambda automates processing, and Amazon S3 is a reliable storage solution.
- D. Correct.
Correct: Amazon Kinesis Data Firehose can both ingest and optionally process data before storing it in Amazon S3, making it a good automated solution.
- E. Incorrect.
Incorrect: Amazon EMR is used for large-scale data processing but is not typically used for ingestion or automated real-time pipelines in this scenario.