DEA-C01 Question 89
Select 2You are tasked with building a data pipeline to extract data from an S3 bucket, transform it using SQL-based queries, and load the processed data into an Amazon Redshift table. The pipeline should be fully serverless and scalable. Which combination of AWS services would you use to design this ETL pipeline?
- A
Amazon S3 for data storage, AWS Glue for transformation, Amazon Redshift for data warehouse
- B
Amazon S3 for data storage, AWS Lambda for transformation, Amazon RDS for data warehouse
- C
Amazon DynamoDB for data storage, AWS Glue for transformation, Amazon Redshift for data warehouse
- D
Amazon S3 for data storage, AWS Glue for transformation, and Amazon Athena for querying the data
- E
Amazon S3 for data storage, AWS Glue for transformation, and Amazon Redshift Spectrum for querying and loading data into Redshift
Show answer and explanation
Correct answers: A, E
Explanation
To build a fully serverless and scalable ETL pipeline, you can use Amazon S3 as the raw data storage layer, AWS Glue for transformations (which supports SQL-based queries and scalable ETL), and Amazon Redshift or Redshift Spectrum for data warehousing and querying. Both combinations in options 1 and 5 are valid as they leverage serverless and scalable AWS services to meet the requirements of the scenario.
- A. Correct.
Correct. Amazon S3 serves as the source for raw data, AWS Glue provides the serverless transformation capabilities using ETL jobs, and Amazon Redshift serves as the scalable data warehouse for loading and querying the processed data.
- B. Incorrect.
Incorrect. While Amazon S3 is suitable for storage, AWS Lambda is not efficient for large-scale SQL-based transformations. Additionally, Amazon RDS is not a data warehouse and does not scale as efficiently as Redshift for analytical workloads.
- C. Incorrect.
Incorrect. Amazon DynamoDB is a NoSQL database and is not suitable for this use case that requires querying structured data and SQL transformations. AWS Glue and Amazon Redshift are the correct choices for transformation and data warehousing, respectively.
- D. Incorrect.
Incorrect. While Amazon S3 and AWS Glue are appropriate choices, Amazon Athena is a query service and does not load data into a data warehouse. This option does not meet the requirement of loading data into Amazon Redshift.
- E. Correct.
Correct. Amazon S3 serves as the raw data source, AWS Glue handles the ETL transformations, and Amazon Redshift Spectrum allows querying S3 data without loading it into Redshift, but it can also be used to load data into Redshift tables.