DEA-C01 Question 159
Select 2You are tasked with designing a data pipeline in AWS to process large volumes of real-time streaming data generated by IoT devices. The data must be ingested, stored temporarily, transformed, and then loaded into a data warehouse for analytics. Which combination of services would best meet these requirements?
- A
Amazon Kinesis Data Streams for data ingestion, AWS Lambda for data transformation, and Amazon Redshift for analytics
- B
Amazon S3 for data ingestion, AWS Glue for data transformation, and Amazon RDS for analytics
- C
Amazon Kinesis Data Firehose for data ingestion, AWS Glue for data transformation, and Amazon Redshift for analytics
- D
Amazon SNS for data ingestion, AWS Lambda for data transformation, and Amazon Aurora for analytics
- E
Amazon Kinesis Data Streams for data ingestion, AWS Glue for data transformation, and Amazon DynamoDB for analytics
Show answer and explanation
Correct answers: A, C
Explanation
For real-time streaming data pipelines, Amazon Kinesis Data Streams or Kinesis Data Firehose are the ideal choices for data ingestion. AWS Lambda or AWS Glue can handle data transformation depending on the specific use case. For analytics, Amazon Redshift is the preferred choice as it is optimized for large-scale data queries and analytics workloads. Combining these services ensures a robust, scalable pipeline for real-time data processing and analytics.
- A. Correct.
Correct: Amazon Kinesis Data Streams is suitable for real-time data ingestion, AWS Lambda can process the streaming data, and Amazon Redshift is a fully-managed data warehouse that supports analytics workloads.
- B. Incorrect.
Incorrect: Amazon S3 is not designed for real-time data ingestion. While AWS Glue can perform transformations, Amazon RDS is not optimized for large-scale analytical workloads.
- C. Correct.
Correct: Amazon Kinesis Data Firehose is a managed service designed for real-time stream ingestion into destinations like Amazon Redshift. AWS Glue can handle ETL transformations, and Amazon Redshift supports analytics.
- D. Incorrect.
Incorrect: Amazon SNS is a pub/sub messaging service, not a real-time ingestion service. While AWS Lambda can process data, Amazon Aurora is a relational database and not optimized for analytical workloads.
- E. Incorrect.
Incorrect: Amazon Kinesis Data Streams can handle real-time ingestion, and AWS Glue can perform transformations, but Amazon DynamoDB is a NoSQL database and not designed for analytical workloads.