DEA-C01 Question 29
Single answerYou are working as a Data Engineer for a retail company. The company wants to ingest batch data from its on-premises database into Amazon S3 for downstream analytics. The data ingestion process must run daily at midnight, and the solution should minimize operational overhead. Which approach would best fulfill this requirement?
- A
Use AWS Glue to create and schedule an ETL job to extract data from the on-premises database and store it in Amazon S3.
- B
Use Amazon Kinesis Data Streams to continuously stream data from the on-premises database to Amazon S3.
- C
Use AWS DataSync to transfer data from the on-premises database to Amazon S3 with a daily schedule.
- D
Manually run a script on an EC2 instance every day at midnight to extract data from the on-premises database and upload it to Amazon S3.
Show answer and explanation
Correct answer: A
Explanation
AWS Glue is the most appropriate tool for this scenario. It allows you to create ETL jobs and schedule them to run at specific times, such as midnight, to automate the batch data ingestion process. This minimizes operational overhead while meeting the daily ingestion requirement. Other options either do not meet the requirements (e.g., real-time streaming with Kinesis) or introduce unnecessary complexity and overhead (e.g., manual scripting).
- A. Correct.
This is the correct option. AWS Glue supports scheduled ETL jobs, making it a suitable choice for batch data ingestion with minimal operational overhead. You can configure the job to run daily at midnight and automatically extract, transform, and load data into Amazon S3.
- B. Incorrect.
This is incorrect. Amazon Kinesis Data Streams is designed for real-time streaming data, not batch ingestion. It does not align with the requirement for daily batch ingestion.
- C. Incorrect.
This is incorrect. While AWS DataSync can transfer data to Amazon S3, it is typically used for file transfers and is not optimized for directly extracting data from an on-premises database.
- D. Incorrect.
This is incorrect. Manually running a script on an EC2 instance introduces significant operational overhead and is not an efficient or scalable solution for scheduled batch ingestion.