DEA-C01 Question 301
Single answerYour company processes customer transaction data daily using an AWS Glue ETL job. Recently, the business team reported inconsistencies in the daily summary report generated by the ETL workflow. Upon investigation, you suspect that some input files were processed multiple times due to a misconfiguration in the workflow. How can you modify the workflow to ensure data is processed exactly once and maintain repeatable business outcomes?
- A
Enable job bookmarks in the AWS Glue ETL job.
- B
Increase the maximum number of retries for the AWS Glue ETL job.
- C
Use Amazon S3 event notifications to trigger the job instead of a scheduled run.
- D
Configure a deduplication step in the ETL job to remove duplicate records.
Show answer and explanation
Correct answer: A
Explanation
Enabling job bookmarks in AWS Glue is the most effective solution to ensure data is processed exactly once. Job bookmarks track the progress of the ETL job, allowing it to process only new or modified data. This prevents the reprocessing of files and ensures consistent and repeatable business outcomes.
- A. Correct.
Enabling job bookmarks in AWS Glue ensures that the ETL job processes only new or modified data since the last successful run, preventing duplicate processing and maintaining repeatable outcomes.
- B. Incorrect.
Increasing the maximum number of retries does not address the issue of duplicate processing; it only impacts job retry behavior in case of failures.
- C. Incorrect.
Using Amazon S3 event notifications to trigger the job changes the scheduling mechanism but does not prevent duplicate processing of files.
- D. Incorrect.
While a deduplication step can help remove duplicate records in the output, it does not address the root cause of the issue, which is the input data being processed multiple times.