DEA-C01 Question 56
Select 3A company processes large amounts of data from IoT devices deployed across multiple regions. The data arrives continuously in real-time and consists of both structured sensor readings and unstructured device logs. Which AWS services and features should the data engineering team use to efficiently handle the volume, velocity, and variety of this data for downstream analysis?
- A
Use Amazon Kinesis Data Streams to process and ingest real-time data from IoT devices.
- B
Store the unstructured logs and structured data directly in Amazon RDS for real-time access.
- C
Use Amazon S3 to store both structured and unstructured data, and leverage S3 events for downstream processing.
- D
Integrate AWS Glue to catalog and transform the data stored in Amazon S3 for analysis.
- E
Leverage Amazon Redshift to process the raw IoT data as it arrives in real-time.
Show answer and explanation
Correct answers: A, C, D
Explanation
To handle the volume, velocity, and variety of IoT data, a combination of services is required. Amazon Kinesis Data Streams efficiently ingests high-velocity data in real-time. Amazon S3 serves as a scalable storage layer for both structured and unstructured data, with S3 events enabling downstream workflows. AWS Glue facilitates data cataloging and transformation to prepare the data for analysis. These services together address the challenges of processing data with high volume, velocity, and variety.
- A. Correct.
Correct: Amazon Kinesis Data Streams is designed to handle high-velocity, real-time streaming data, making it an ideal choice for ingesting real-time IoT data.
- B. Incorrect.
Incorrect: Amazon RDS is not optimized for handling high-velocity or unstructured data like IoT device logs. It is better suited for transactional workloads with structured data.
- C. Correct.
Correct: Amazon S3 is a cost-effective and scalable storage solution for both structured and unstructured data. Additionally, S3 events can be used to trigger downstream processing workflows.
- D. Correct.
Correct: AWS Glue provides data cataloging and ETL capabilities, making it suitable for preparing and transforming diverse datasets stored in S3 for analysis.
- E. Incorrect.
Incorrect: Amazon Redshift is a data warehouse designed for batch analytics and not optimized for processing raw real-time data streams.