DEA-C01 Question 5
Select 3You are a Data Engineer tasked with building a data ingestion pipeline for a client that collects real-time clickstream data from their website. The client requires the data to be processed with minimal latency and stored in a data lake for future analytics. Which combination of AWS services would you use to build this pipeline?
- A
Amazon Kinesis Data Streams for real-time data ingestion
- B
AWS Glue for real-time data ingestion
- C
Amazon S3 as the data lake storage
- D
Amazon Redshift for data lake storage
- E
Amazon Kinesis Data Firehose for delivering data to the data lake
Show answer and explanation
Correct answers: A, C, E
Explanation
To meet the client's requirements of real-time ingestion and storing data in a data lake, Amazon Kinesis Data Streams handles the real-time clickstream data ingestion, Amazon Kinesis Data Firehose delivers the data to Amazon S3, and Amazon S3 serves as the data lake. This combination ensures minimal latency and scalable data storage for future analytics.
- A. Correct.
Amazon Kinesis Data Streams is designed for real-time data ingestion, making it suitable for handling clickstream data with minimal latency.
- B. Incorrect.
AWS Glue is primarily used for ETL (Extract, Transform, Load) tasks and not optimized for real-time data ingestion, so it is not suitable for this use case.
- C. Correct.
Amazon S3 is a scalable and cost-effective storage option for a data lake, meeting the client's requirements.
- D. Incorrect.
Amazon Redshift is a data warehouse solution and is not typically used as a data lake. It is optimized for analytics rather than scalable storage for raw data.
- E. Correct.
Amazon Kinesis Data Firehose can be used to deliver streaming data into Amazon S3, ensuring seamless integration with the data lake.