DEA-C01 Question 482
Select 3A data engineering team is tasked with designing a data pipeline that ingests real-time streaming data from IoT devices into AWS for analysis. The data should be processed with low latency and stored in a data warehouse for querying with SQL. Which combination of AWS services should the team use to meet these requirements?
- A
Amazon Kinesis Data Streams for ingesting real-time data and Amazon Redshift for querying
- B
Amazon S3 for ingesting real-time data and AWS Glue for processing
- C
Amazon Kinesis Data Firehose for ingesting real-time data and Amazon Redshift for querying
- D
Amazon Kinesis Data Analytics for processing streaming data and Amazon Redshift for querying
- E
Amazon EMR for ingesting real-time data and Amazon Athena for querying
Show answer and explanation
Correct answers: A, C, D
Explanation
To handle real-time streaming data from IoT devices, AWS offers services like Amazon Kinesis Data Streams, Kinesis Data Firehose, and Kinesis Data Analytics for data ingestion and real-time processing. The processed data can then be stored in Amazon Redshift, a fully managed data warehouse, for running SQL queries. This combination ensures low latency and efficient querying in the pipeline. Options involving Amazon S3 or Amazon EMR are not suitable for the real-time requirements described in the scenario.
- A. Correct.
Correct. Amazon Kinesis Data Streams is designed for real-time data ingestion, and Amazon Redshift is suitable for running SQL queries on structured data stored in a data warehouse.
- B. Incorrect.
Incorrect. Amazon S3 is not suitable for real-time data ingestion as it is a storage service primarily used for batch data processing.
- C. Correct.
Correct. Amazon Kinesis Data Firehose can ingest real-time data and directly load it into Amazon Redshift for SQL-based analysis.
- D. Correct.
Correct. Amazon Kinesis Data Analytics can process streaming data in real-time, and Amazon Redshift allows for efficient querying of processed data.
- E. Incorrect.
Incorrect. Amazon EMR is typically used for big data processing using frameworks like Apache Spark or Hadoop, not for real-time data ingestion. Additionally, Amazon Athena is used for querying data in S3, which is not part of this scenario.