DEA-C01 Question 297
Select 3You are tasked with designing a data pipeline in AWS to process large volumes of clickstream data in near real-time. The incoming data must be ingested, processed, and stored for analytics. The solution should be scalable, fault-tolerant, and able to handle out-of-order events. Which services and configurations should you use to meet these requirements?
- A
Use Amazon Kinesis Data Streams for ingestion and buffering of clickstream data.
- B
Use Amazon S3 with EventBridge to directly ingest and store clickstream data.
- C
Use AWS Lambda to process records from Kinesis Data Streams and write the processed data to Amazon S3.
- D
Use Amazon Redshift for direct ingestion of clickstream data for analytics without preprocessing.
- E
Enable the enhanced fan-out feature in Kinesis Data Streams to improve parallel processing by downstream consumers.
Show answer and explanation
Correct answers: A, C, E
Explanation
To build a scalable, fault-tolerant, and near real-time data pipeline for clickstream data, Amazon Kinesis Data Streams is used for ingestion and buffering. AWS Lambda processes the data and writes it to Amazon S3 for storage and analytics. The enhanced fan-out feature in Kinesis ensures efficient parallel processing by multiple consumers, which is critical for handling high-throughput, real-time data. The other options, such as direct ingestion into Amazon Redshift or using S3 with EventBridge, do not meet the real-time requirements of this use case.
- A. Correct.
Amazon Kinesis Data Streams is a suitable service for ingesting and buffering large volumes of real-time data, making it a correct choice for handling clickstream data.
- B. Incorrect.
Amazon S3 with EventBridge is not designed for real-time ingestion of clickstream data. It is more suitable for batch-oriented workflows.
- C. Correct.
AWS Lambda can process records from Kinesis Data Streams in near real-time and write the processed data to Amazon S3, making it a correct choice for processing and storing the data for analytics.
- D. Incorrect.
Amazon Redshift is not designed to directly ingest high-velocity clickstream data in real-time. It is more appropriate for structured analytical queries on already processed and stored data.
- E. Correct.
The enhanced fan-out feature in Kinesis Data Streams allows multiple consumers to process data in parallel without competing for read throughput, which is beneficial for scaling and fault tolerance in real-time data pipelines.