DEA-C01 Question 484
Single answerA data engineering team is designing a data pipeline to process millions of daily log files from a web application. They need to store the raw log data in a scalable and cost-efficient manner, while ensuring that the storage service supports integration with AWS Glue for data cataloging and ETL jobs. Which storage solution should they choose?
- A
Amazon RDS
- B
Amazon EFS
- C
Amazon S3
- D
Amazon DynamoDB
Show answer and explanation
Correct answer: C
Explanation
Amazon S3 is the most suitable storage solution for this scenario because it is highly scalable and cost-efficient for storing large amounts of raw log data. Additionally, it integrates with AWS Glue, enabling the team to catalog and process the data effectively. Other options, such as RDS, EFS, and DynamoDB, are either not cost-efficient, not optimized for this use case, or lack seamless integration with AWS Glue for this specific requirement.
- A. Incorrect.
Amazon RDS is a relational database service, which is not suitable for storing raw log files due to its limitations in scalability and cost for this use case.
- B. Incorrect.
Amazon EFS is a file storage service for use with Amazon EC2 instances, and while it is scalable, it is not as cost-efficient or optimized for large-scale log storage compared to Amazon S3.
- C. Correct.
Amazon S3 is a highly scalable, durable, and cost-efficient object storage service. It integrates seamlessly with AWS Glue for cataloging and processing data, making it ideal for storing raw log files.
- D. Incorrect.
Amazon DynamoDB is a NoSQL database service designed for low-latency, high-throughput workloads. It is not intended for storing raw log files and lacks native integration with AWS Glue for data cataloging.