DEA-C01 Question 71
Single answerYou are designing a data pipeline in AWS to process large volumes of transactional data from an on-premises database. The data is extracted daily, transformed, and loaded into an Amazon Redshift cluster for analytics. To handle intermediate data during the ETL process, which of the following is the most appropriate intermediate data staging location?
- A
Amazon S3 with appropriate bucket policies
- B
Amazon RDS with read replicas
- C
Amazon DynamoDB with on-demand capacity
- D
AWS Lambda temporary storage
Show answer and explanation
Correct answer: A
Explanation
Amazon S3 is the most appropriate intermediate data staging location in this scenario due to its scalability, cost-effectiveness, and ability to handle large datasets with various formats. It integrates seamlessly with ETL tools like AWS Glue and supports use cases such as intermediate storage during data transformations.
- A. Correct.
Amazon S3 is a highly durable and cost-effective storage service suitable for intermediate data staging. It allows for parallel processing, supports multiple data formats, and integrates well with services like AWS Glue and Amazon Redshift.
- B. Incorrect.
Amazon RDS is a managed relational database service and is not optimized for intermediate data staging in ETL pipelines. It is better suited for database storage rather than handling temporary staging data.
- C. Incorrect.
Amazon DynamoDB is a NoSQL database service designed for low-latency, high-speed applications. It is not ideal for intermediate data staging due to its pricing model and lack of integration with ETL workflows.
- D. Incorrect.
AWS Lambda's temporary storage is limited to 512 MB and is not designed for storing large volumes of intermediate data in ETL processes.