DEA-C01 Question 20
Select 2A data engineering team is tasked with designing a solution to ingest sensor data from IoT devices for real-time analytics and historical storage. The data arrives in small batches every 10 seconds and must be processed with minimal latency. Additionally, the team needs to ensure that historical data is stored for long-term analysis. Which combination of AWS services best supports this ingestion pattern?
- A
Amazon Kinesis Data Streams for real-time ingestion and Amazon S3 for long-term storage
- B
Amazon SQS for real-time ingestion and Amazon RDS for long-term storage
- C
AWS IoT Core for real-time ingestion and Amazon DynamoDB for long-term storage
- D
Amazon MSK (Managed Streaming for Apache Kafka) for real-time ingestion and Amazon S3 for long-term storage
- E
Amazon Kinesis Data Firehose for real-time ingestion and Amazon Redshift for long-term storage
Show answer and explanation
Correct answers: A, D
Explanation
For real-time ingestion with minimal latency, Amazon Kinesis Data Streams or Amazon MSK are excellent choices as they handle high-throughput streaming data efficiently. For long-term storage, Amazon S3 is highly scalable and cost-effective, making it ideal for storing historical data. Together, these combinations meet both the real-time and historical data requirements of the use case.
- A. Correct.
Correct. Amazon Kinesis Data Streams is designed for real-time ingestion of streaming data, and Amazon S3 provides scalable, durable storage for historical data.
- B. Incorrect.
Incorrect. While Amazon SQS can handle messages, it is not optimized for real-time ingestion of streaming data like Kinesis Data Streams. Amazon RDS is not typically used for large-scale historical data storage.
- C. Incorrect.
Incorrect. AWS IoT Core is suitable for managing IoT device communication but not for general-purpose real-time ingestion. DynamoDB, while suitable for key-value or document-based storage, is not optimized for large-scale historical data storage.
- D. Correct.
Correct. Amazon MSK (Managed Streaming for Apache Kafka) is a robust alternative for real-time ingestion of streaming data, and Amazon S3 is suitable for cost-effective, long-term data storage.
- E. Incorrect.
Incorrect. Amazon Kinesis Data Firehose is suitable for real-time data delivery but is not designed for processing streaming data with minimal latency. Amazon Redshift, while great for data warehousing, is not ideal for storing raw historical data directly.