DEA-C01 Question 156
Select 2You are designing a data processing pipeline on AWS to analyze large-scale streaming data from IoT devices. The data needs to be processed in near real-time and stored for long-term analytics. Which combination of services would be most appropriate for this use case?
- A
Amazon Kinesis Data Streams for ingesting data, AWS Lambda for processing, and Amazon S3 for storage
- B
Amazon SQS for ingesting data, AWS Glue for processing, and Amazon Redshift for storage
- C
Amazon Kinesis Data Firehose for ingesting data, Amazon EMR for processing, and Amazon S3 for storage
- D
Amazon MSK (Managed Streaming for Apache Kafka) for ingesting data, AWS Batch for processing, and Amazon RDS for storage
- E
Amazon Kinesis Data Streams for ingesting data, Amazon Kinesis Data Analytics for processing, and Amazon S3 for storage
Show answer and explanation
Correct answers: A, E
Explanation
For near real-time processing of large-scale streaming data, Amazon Kinesis Data Streams is a reliable option for ingesting data, as it supports high-throughput and low-latency processing. AWS Lambda or Amazon Kinesis Data Analytics can process the data in near real-time depending on the complexity of the operation. Finally, Amazon S3 is a cost-effective and scalable solution for storing processed data for long-term analytics. The correct combinations are 1 (Kinesis Data Streams + Lambda + S3) and 5 (Kinesis Data Streams + Kinesis Data Analytics + S3).
- A. Correct.
Amazon Kinesis Data Streams can handle large-scale streaming data ingestion, AWS Lambda can process data in near real-time, and Amazon S3 is well-suited for long-term storage. This is a highly recommended architecture for streaming use cases.
- B. Incorrect.
Amazon SQS is not optimized for high-throughput streaming data ingestion, and AWS Glue is meant for ETL processes, not near real-time processing. Amazon Redshift is a data warehouse and not ideal for the initial stages of processing large-scale streaming data.
- C. Incorrect.
Amazon Kinesis Data Firehose is suitable for streaming data ingestion, and Amazon EMR can process data at scale. However, EMR is not designed for low-latency near real-time processing. Amazon S3 is a good choice for storage.
- D. Incorrect.
Amazon MSK is a managed Kafka service suited for streaming data ingestion, but AWS Batch is designed for batch processing, not real-time use cases. Amazon RDS is a relational database and not ideal for storing large-scale streaming data.
- E. Correct.
Amazon Kinesis Data Streams is a strong choice for high-throughput streaming data ingestion. Amazon Kinesis Data Analytics processes data in near real-time, and Amazon S3 is an excellent option for long-term storage.