DEA-C01 Question 514
Select 2You are designing a data pipeline using AWS services to handle real-time streaming data from IoT devices. The pipeline must process data in near real-time, store raw data for compliance, and enable querying processed data for analytics. Which combination of services would best meet these requirements?
- A
Amazon Kinesis Data Streams for ingesting data, AWS Lambda for processing, Amazon S3 for storing raw data, and Amazon Redshift for querying processed data.
- B
Amazon SQS for ingesting data, AWS Glue for processing, Amazon DynamoDB for storing raw data, and Amazon EMR for querying processed data.
- C
Amazon Kinesis Data Firehose for ingesting and transforming data, Amazon S3 for storing raw data, and Amazon Athena for querying processed data.
- D
Amazon Kafka for ingesting data, Amazon S3 for storing raw data, AWS Glue for transforming data, and Amazon QuickSight for querying processed data.
- E
Amazon Kinesis Data Streams for ingesting data, AWS Lambda for processing, Amazon DynamoDB for storing raw data, and Amazon Aurora for querying processed data.
Show answer and explanation
Correct answers: A, C
Explanation
When designing for real-time streaming data pipelines, Amazon Kinesis services (Data Streams or Firehose) are optimal for ingesting the data. For storage, Amazon S3 is a scalable and cost-effective option for storing raw data. AWS Lambda can enable near real-time processing. For analytics, Amazon Redshift or Athena are well-suited for querying and analyzing processed data. The combination in Options 1 and 3 covers all these requirements effectively.
- A. Correct.
This option is correct. Amazon Kinesis Data Streams can handle real-time streaming data ingestion, AWS Lambda can process data in near real-time, Amazon S3 is ideal for storing raw data, and Amazon Redshift allows for efficient analytics queries.
- B. Incorrect.
This option is incorrect. Amazon SQS is not designed for real-time streaming data ingestion, and DynamoDB is not a suitable choice for storing large volumes of raw data.
- C. Correct.
This option is correct. Amazon Kinesis Data Firehose simplifies ingesting and transforming streaming data, Amazon S3 provides a scalable solution for storing raw data, and Amazon Athena allows querying data directly in S3 using SQL.
- D. Incorrect.
This option is incorrect. While Amazon Kafka can handle data ingestion and QuickSight is useful for visualization, AWS Glue and QuickSight together do not provide the necessary near real-time data processing and querying capabilities.
- E. Incorrect.
This option is incorrect. While Kinesis Data Streams and AWS Lambda are suitable for ingestion and processing, DynamoDB is not ideal for storing raw data, and Aurora is not optimized for querying high-volume analytics workloads.