DEA-C01 Question 539
Select 2You are working as a data engineer for a company that processes large volumes of real-time streaming data from IoT devices. The data needs to be ingested, processed, and stored in near real-time for analytics. The solution must be highly available, scalable, and cost-effective. Which combination of AWS services would best meet these requirements?
- A
Amazon Kinesis Data Streams for ingestion, AWS Lambda for processing, and Amazon S3 for storage
- B
Amazon SQS for ingestion, Amazon EC2 for processing, and Amazon RDS for storage
- C
Amazon Kinesis Firehose for ingestion and processing, and Amazon Redshift for storage
- D
Amazon Managed Streaming for Apache Kafka (MSK) for ingestion, AWS Glue for processing, and Amazon S3 for storage
- E
Amazon Kinesis Data Analytics for both ingestion and processing, and Amazon DynamoDB for storage
Show answer and explanation
Correct answers: A, D
Explanation
The best solutions for real-time streaming data involve using services like Amazon Kinesis Data Streams or Amazon Managed Streaming for Apache Kafka (MSK) for ingestion, combined with processing services like AWS Lambda or AWS Glue for ETL tasks, and cost-effective storage solutions like Amazon S3. These combinations ensure scalability, high availability, and cost efficiency while meeting the requirements for real-time data analytics.
- A. Correct.
Correct: Amazon Kinesis Data Streams provides scalable and durable ingestion for real-time streaming data, AWS Lambda can process the data in real-time, and Amazon S3 is a cost-effective and reliable storage solution.
- B. Incorrect.
Incorrect: Amazon SQS is not ideal for high-throughput real-time streaming ingestion, and Amazon RDS is not optimized for high-volume analytics storage.
- C. Incorrect.
Partially correct: While Amazon Kinesis Firehose can handle ingestion and processing, Amazon Redshift is better suited for analytics rather than near real-time storage. This combination may not fully address the requirement for near real-time processing.
- D. Correct.
Correct: Amazon MSK is a managed Apache Kafka service ideal for real-time ingestion, AWS Glue can process data for ETL purposes, and Amazon S3 is suitable for cost-effective storage.
- E. Incorrect.
Incorrect: Amazon Kinesis Data Analytics is a processing service, not an ingestion service, and Amazon DynamoDB is more suited for low-latency key-value data queries rather than analytics storage.