DEA-C01 Question 55
Select 2A retail company wants to create an ETL pipeline to process daily sales data stored in Amazon S3 and load it into Amazon Redshift for analytics. The pipeline must ensure scalability, fault tolerance, and minimal operational overhead. Which combination of services would best meet these requirements?
- A
Use AWS Glue to create ETL jobs to extract data from Amazon S3, transform it, and load it into Amazon Redshift.
- B
Use Amazon EMR with Apache Spark to build a custom ETL pipeline for processing and loading data into Amazon Redshift.
- C
Use Amazon Athena to query the data in Amazon S3 and directly visualize results in Amazon QuickSight.
- D
Use AWS Data Pipeline to schedule and automate the extraction, transformation, and loading into Amazon Redshift.
- E
Use Amazon Kinesis Data Firehose to directly stream data from Amazon S3 into Amazon Redshift.
Show answer and explanation
Correct answers: A, D
Explanation
For a scalable, fault-tolerant ETL pipeline with minimal operational overhead, AWS Glue is an excellent choice due to its managed nature and integration with Amazon Redshift. AWS Data Pipeline can also complement the workflow by automating and scheduling ETL processes. Other options like Amazon EMR and Kinesis Data Firehose require more management or are not designed for the specific use case of batch processing from Amazon S3 to Amazon Redshift.
- A. Correct.
AWS Glue is a fully managed ETL service that can efficiently handle the extraction, transformation, and loading of data into Amazon Redshift with minimal operational overhead.
- B. Incorrect.
Amazon EMR with Apache Spark is suitable for custom ETL jobs, but it requires more operational effort to manage and scale the infrastructure, making it less ideal for this use case.
- C. Incorrect.
Amazon Athena is a serverless query service, but it does not perform ETL operations or load data into Amazon Redshift. It is more suited for ad-hoc querying of data directly in Amazon S3.
- D. Correct.
AWS Data Pipeline can be used to schedule and automate ETL tasks, and it integrates well with Amazon Redshift, making it a viable option for this use case.
- E. Incorrect.
Amazon Kinesis Data Firehose is used for streaming data directly into Amazon Redshift but is not designed for batch ETL jobs from Amazon S3.