DEA-C01 Question 422
Select 3You are tasked with building a scalable data pipeline on AWS to process large volumes of real-time streaming data from IoT sensors. The processed data will be stored in Amazon S3 for further analysis and utilized by a machine learning algorithm. Which combination of AWS services should you use to efficiently handle the ingestion, processing, and storage of this data?
- A
Amazon Kinesis Data Streams for ingestion, AWS Lambda for processing, and Amazon S3 for storage
- B
Amazon SQS for ingestion, AWS Glue for processing, and Amazon DynamoDB for storage
- C
Amazon Kinesis Firehose for ingestion, AWS Lambda for processing, and Amazon S3 for storage
- D
Amazon Kinesis Data Streams for ingestion, Amazon EMR for processing, and Amazon S3 for storage
- E
Amazon SNS for ingestion, AWS Glue for processing, and Amazon Redshift for storage
Show answer and explanation
Correct answers: A, C, D
Explanation
To handle real-time streaming data from IoT sensors, Amazon Kinesis (Data Streams or Firehose) is the most suitable service for ingestion. For real-time processing, AWS Lambda or Amazon EMR can be used depending on the complexity of processing needs. Amazon S3 is the preferred storage service for scalable and cost-effective storage of processed data.
- A. Correct.
Amazon Kinesis Data Streams is a great fit for real-time ingestion of streaming data. AWS Lambda can process the data in near real-time, and Amazon S3 is a cost-effective storage solution. This combination is efficient for scalable and serverless architectures.
- B. Incorrect.
Amazon SQS is typically used for message queuing rather than high-throughput real-time streaming. AWS Glue is designed for ETL processes and not real-time processing. DynamoDB is optimized for NoSQL database use cases, not large-scale data storage.
- C. Correct.
Amazon Kinesis Firehose simplifies the ingestion process and can directly deliver streamed data to Amazon S3. AWS Lambda can process the data in real-time, and Amazon S3 serves as a scalable storage solution.
- D. Correct.
Amazon Kinesis Data Streams supports high-throughput real-time ingestion. Amazon EMR provides a powerful distributed processing framework (e.g., Apache Spark) for large-scale data processing. Amazon S3 is an ideal storage service for data lakes.
- E. Incorrect.
Amazon SNS is a pub-sub messaging service and not suitable for high-throughput real-time ingestion. AWS Glue is designed for batch ETL workloads, and Amazon Redshift is a data warehouse optimized for analytical queries, not raw data storage.