DEA-C01 Question 10
Select 3You are designing a data pipeline for a retail company that needs to process clickstream data in near real-time. The processed data will be used for customer behavior analysis and stored in a data warehouse for further querying. Which AWS services should you use to build this pipeline efficiently and cost-effectively?
- A
Amazon Kinesis Data Streams for real-time data ingestion
- B
AWS Glue for real-time data transformation
- C
Amazon Redshift for storing the processed data
- D
Amazon S3 for storing raw and processed data
- E
Amazon EMR for running Apache Spark jobs in real-time
Show answer and explanation
Correct answers: A, C, D
Explanation
To process clickstream data in near real-time, Amazon Kinesis Data Streams can be used for data ingestion. Processed data can be stored in Amazon Redshift to enable efficient querying for customer behavior analysis. Amazon S3 is ideal for storing both raw and processed data due to its scalability and cost-effectiveness. AWS Glue and Amazon EMR are not optimal for this pipeline as they are more suited for batch processing rather than real-time processing.
- A. Correct.
Amazon Kinesis Data Streams is designed for real-time data ingestion, making it suitable for processing clickstream data in near real-time.
- B. Incorrect.
AWS Glue is primarily used for ETL processes and does not support true real-time transformations. It is designed for batch transformations.
- C. Correct.
Amazon Redshift is a data warehouse service optimized for analytical queries, making it ideal for storing processed data for customer behavior analysis.
- D. Correct.
Amazon S3 is a cost-effective option for storing raw clickstream data and processed data, enabling a durable and scalable storage layer.
- E. Incorrect.
Amazon EMR is better suited for large-scale batch processing and is not the most efficient option for real-time data processing.