DEA-C01 Question 87
Select 2You are designing an ETL pipeline on AWS to process large volumes of clickstream data from an S3 bucket, transform it, and load the results into an Amazon Redshift cluster for reporting. The pipeline needs to be automated and scalable with minimal operational overhead. Which combination of services should you use to implement this pipeline effectively?
- A
Amazon S3 for data storage, AWS Glue for ETL, and Amazon Redshift for data warehousing
- B
Amazon Kinesis Data Streams for data ingestion, AWS Lambda for transformation, and Amazon DynamoDB for storage
- C
Amazon S3 for data storage, Amazon EMR for transformation, and Amazon Redshift for data warehousing
- D
Amazon S3 for data storage, AWS Data Pipeline for orchestration, and Amazon RDS for data warehousing
- E
Amazon S3 for data storage, AWS Glue for cataloging, and Amazon QuickSight for visualization
Show answer and explanation
Correct answers: A, C
Explanation
To create a scalable ETL pipeline for processing large volumes of data, Amazon S3 serves as the data storage layer, AWS Glue or Amazon EMR can perform the data transformation, and Amazon Redshift is an ideal choice for a data warehouse. AWS Glue provides serverless ETL capabilities, while Amazon EMR offers flexibility with distributed processing frameworks. Both approaches integrate well with Amazon S3 and Redshift, making them suitable for building automated pipelines with minimal operational overhead.
- A. Correct.
Correct. Amazon S3 can store the raw data, AWS Glue can automate the ETL process (extract, transform, and load), and Amazon Redshift can serve as the data warehouse for reporting.
- B. Incorrect.
Incorrect. While Amazon Kinesis is suitable for real-time data ingestion, this scenario involves processing static data from S3. DynamoDB is not a data warehousing solution.
- C. Correct.
Correct. Amazon S3 can store the raw data, Amazon EMR can process and transform the data using distributed frameworks like Spark, and Amazon Redshift can serve as the data warehouse.
- D. Incorrect.
Incorrect. AWS Data Pipeline is outdated for orchestrating modern ETL pipelines, and Amazon RDS is not designed for large-scale data warehousing.
- E. Incorrect.
Incorrect. While this setup can be used for data cataloging and visualization, it does not fulfill the ETL pipeline requirements for transformation and loading data into a warehouse.