DEA-C01 Question 90
Select 2You are tasked with building an ETL pipeline that ingests raw log data into an S3 bucket, processes it to extract specific metrics, and loads the processed data into a Redshift data warehouse for analytical queries. The pipeline should handle large-scale data, support incremental data processing, and minimize operational overhead. Which combination of AWS services would best meet this requirement?
- A
Amazon S3, AWS Glue, and Amazon Redshift
- B
Amazon S3, AWS Lambda, and Amazon RDS
- C
Amazon Kinesis Data Streams, AWS Glue, and Amazon Redshift
- D
Amazon S3, Amazon EMR, and Amazon Redshift
- E
Amazon DynamoDB, AWS Glue, and Amazon Redshift
Show answer and explanation
Correct answers: A, C
Explanation
Building scalable and cost-effective ETL pipelines requires choosing the right combination of services. Amazon S3 is ideal for storing raw data, AWS Glue provides a serverless ETL solution for data transformation, and Amazon Redshift enables efficient analytical workloads. For real-time data ingestion, Amazon Kinesis Data Streams is a suitable choice, while AWS Glue and Redshift ensure the pipeline can handle transformation and analytics, respectively. This combination minimizes operational overhead and meets the requirements of incremental data processing.
- A. Correct.
Correct. Amazon S3 is used for raw data storage, AWS Glue provides a serverless transformation service to process the data, and Amazon Redshift serves as the destination for analytics. This combination is cost-effective and scalable.
- B. Incorrect.
Incorrect. While Amazon S3 and AWS Lambda can be used for data processing, Amazon RDS is not designed for large-scale analytical workloads like Amazon Redshift.
- C. Correct.
Correct. Amazon Kinesis Data Streams can be used for real-time data ingestion, AWS Glue for data transformations, and Amazon Redshift for analytical queries. This setup is suitable for pipelines requiring real-time or near-real-time processing.
- D. Incorrect.
Incorrect. Amazon EMR can process data at scale, but it requires more operational management compared to AWS Glue. For a serverless solution with minimal overhead, AWS Glue is a better choice.
- E. Incorrect.
Incorrect. Amazon DynamoDB is a NoSQL database and not suitable for ETL pipelines designed to process and analyze large-scale data. It is not a replacement for Amazon S3 or Redshift in this scenario.