DEA-C01 Question 305
Select 2Your organization processes financial transaction data using AWS Glue. Recently, there have been inconsistencies in the data outputs, leading to errors in downstream analytics. To ensure repeatable and reliable data processing, you need to implement measures to identify issues early and improve the consistency of the data pipeline. Which of the following actions should you take? (Select TWO)
- A
Implement AWS Glue job bookmarks to ensure idempotent processing of data.
- B
Enable AWS CloudTrail logs to monitor all API activities in the data pipeline.
- C
Use AWS Glue Workflows to define, track, and monitor the sequence of data processing steps.
- D
Implement AWS Glue Data Quality rules to validate data at various stages of the pipeline.
- E
Use Amazon QuickSight to visualize inconsistencies in the data processing pipeline in real-time.
Show answer and explanation
Correct answers: A, D
Explanation
To ensure repeatable and consistent data processing, it is important to implement mechanisms that prevent processing errors and validate the quality of incoming or processed data. Job bookmarks in AWS Glue help maintain idempotency, ensuring that data is not reprocessed unnecessarily. AWS Glue Data Quality rules provide the ability to validate and troubleshoot inconsistencies in the data pipeline, allowing for early detection and resolution of issues. These measures work together to maintain reliable and repeatable business outcomes.
- A. Correct.
Implementing AWS Glue job bookmarks ensures that the data pipeline only processes new or unprocessed data, which helps maintain consistency and idempotency in repeatable processing.
- B. Incorrect.
While AWS CloudTrail can help monitor API calls, it does not directly address inconsistencies in data outputs or processing logic.
- C. Incorrect.
AWS Glue Workflows are useful for orchestrating and monitoring the steps in a pipeline but do not detect or resolve issues causing data inconsistencies.
- D. Correct.
AWS Glue Data Quality rules allow you to define and enforce constraints on your data, which helps identify and troubleshoot inconsistencies early in the pipeline.
- E. Incorrect.
Amazon QuickSight is a visualization tool but does not directly contribute to maintaining or troubleshooting data processing pipelines.