DEA-C01 Question 420
Select 2You are working as a Data Engineer for a company that processes large amounts of streaming data from IoT devices. The data must be ingested, processed, and stored in real-time. The company requires scalability, low latency, and the ability to perform complex analytics in near real-time. Which combination of AWS services would be the most appropriate to design this solution?
- A
Amazon Kinesis Data Streams for ingestion, AWS Lambda for processing, and Amazon S3 for storage
- B
Amazon Kinesis Data Firehose for ingestion, Amazon EMR for processing, and Amazon RDS for storage
- C
Amazon Kinesis Data Streams for ingestion, Amazon Kinesis Data Analytics for processing, and Amazon Redshift for storage
- D
Amazon SQS for ingestion, AWS Glue for processing, and Amazon DynamoDB for storage
- E
Amazon Kinesis Data Firehose for ingestion, AWS Lambda for processing, and Amazon Elasticsearch Service for storage and analytics
Show answer and explanation
Correct answers: A, C
Explanation
For real-time data processing pipelines, Amazon Kinesis Data Streams is a highly scalable solution for ingestion. AWS Lambda or Amazon Kinesis Data Analytics can handle real-time processing, depending on the complexity of the analytics required. Amazon S3 and Amazon Redshift are both suitable storage options, with S3 providing cost-effective object storage and Redshift supporting analytical queries on structured data. The selected combinations ensure low latency, scalability, and the ability to perform complex analytics in near real-time.
- A. Correct.
This is a valid choice. Amazon Kinesis Data Streams can handle real-time ingestion with low latency, AWS Lambda provides serverless compute for processing the data, and Amazon S3 is a cost-effective and scalable storage solution.
- B. Incorrect.
This is not the most appropriate choice, as Amazon EMR is better suited for batch processing rather than real-time stream processing. Additionally, Amazon RDS is not designed for handling high-volume, real-time data ingestion.
- C. Correct.
This is a valid choice. Amazon Kinesis Data Streams is ideal for real-time data ingestion, Amazon Kinesis Data Analytics can perform complex real-time analytics, and Amazon Redshift is suitable for analytical queries on processed data.
- D. Incorrect.
This is not an ideal choice. While Amazon SQS is a message queue service, it is not optimized for real-time streaming ingestion. Additionally, AWS Glue is primarily used for ETL operations rather than real-time processing.
- E. Incorrect.
This is not the best choice. While Amazon Kinesis Data Firehose can handle data ingestion, AWS Lambda may not be suitable for all real-time processing needs due to its invocation limits, and Amazon Elasticsearch Service is better suited for search and log analytics rather than general-purpose analytics.