DEA-C01 Question 27
Single answerA company needs to ingest large volumes of log data from its on-premises servers into Amazon S3 for further processing. The log files are generated hourly and must be transferred to S3 in batches every 6 hours. Additionally, the process should be automated and resilient to failures. Which solution would best meet these requirements?
- A
Use AWS Glue to schedule a job that pulls log files from the on-premises servers and writes them to Amazon S3.
- B
Use Amazon Kinesis Data Streams to continuously stream log data to Amazon S3.
- C
Set up an AWS DataSync task to transfer the log files from the on-premises servers to Amazon S3, and schedule the task to run every 6 hours.
- D
Use AWS Snowball to transfer the log data to Amazon S3 every 6 hours.
Show answer and explanation
Correct answer: C
Explanation
AWS DataSync is the best choice for this use case because it provides a managed service for automating and scheduling data transfers between on-premises servers and AWS storage services like S3. It is well-suited for handling batch data ingestion on a recurring schedule, such as every 6 hours, and includes built-in resiliency to handle failures. Other options, such as AWS Glue and Amazon Kinesis Data Streams, are not designed for this specific batch ingestion scenario, and AWS Snowball is only suitable for offline, large-scale data migrations.
- A. Incorrect.
AWS Glue is primarily used for ETL (Extract, Transform, Load) operations and does not natively support pulling files from on-premises servers for scheduled batch ingestion. This makes it unsuitable for the given use case.
- B. Incorrect.
Amazon Kinesis Data Streams is designed for real-time streaming data ingestion, not for batch ingestion at fixed intervals. It is not a cost-effective or appropriate solution for this scenario.
- C. Correct.
AWS DataSync is specifically designed for transferring data between on-premises servers and AWS storage services like S3. It supports scheduling tasks and is resilient to failures, making it an ideal solution for this use case.
- D. Incorrect.
AWS Snowball is intended for transferring large-scale data in offline scenarios and is not suitable for a recurring, automated batch ingestion process that occurs every 6 hours.