DEA-C01 Question 536
Select 2A company wants to build a data pipeline to process streaming data from IoT devices in real-time and store it in a data lake for further analysis. The data needs to be transformed during ingestion to ensure consistency. Which combination of AWS services is most suitable for this use case?
- A
Amazon Kinesis Data Streams for ingesting the data, AWS Lambda for transformation, and Amazon S3 for data lake storage
- B
AWS Glue for ingesting, transforming, and storing the data in a data lake
- C
Amazon Kinesis Data Firehose for ingesting and transforming the data, and Amazon S3 for data lake storage
- D
Amazon Redshift for ingesting the data and transforming it during storage
- E
Amazon EMR for ingesting, transforming, and storing the data in a data lake
Show answer and explanation
Correct answers: A, C
Explanation
To process streaming data in real-time and store it in a data lake, you need services that support streaming ingestion, real-time transformation, and scalable storage. Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose are ideal for ingesting streaming data. AWS Lambda enables real-time transformation, and Amazon S3 provides a scalable and cost-effective data lake storage solution. AWS Glue and Amazon EMR are more suitable for batch processing, while Amazon Redshift is a data warehouse, not designed for real-time ingestion.
- A. Correct.
Correct. Amazon Kinesis Data Streams can ingest streaming data, AWS Lambda can transform it in real-time, and Amazon S3 is a scalable and cost-effective storage solution for a data lake.
- B. Incorrect.
Incorrect. AWS Glue is primarily a serverless ETL tool for batch processing, not suitable for real-time ingestion and transformation of streaming data.
- C. Correct.
Correct. Amazon Kinesis Data Firehose provides managed delivery of streaming data to destinations like S3, and it supports real-time data transformation using AWS Lambda.
- D. Incorrect.
Incorrect. Amazon Redshift is a data warehouse, not designed for real-time ingestion and transformation of streaming data.
- E. Incorrect.
Incorrect. Amazon EMR is a big data processing service, but it is not specifically optimized for real-time ingestion and transformation of streaming data.