DEA-C01 Question 269
Single answerA financial organization is building a data pipeline for processing sensitive customer transactions using AWS Glue. To ensure accuracy and trustworthiness of the data, the organization wants to track data transformations, understand the origin of the data, and audit the data flow end-to-end. Which approach will help meet these requirements effectively?
- A
Enable AWS Glue Data Catalog to record metadata about the data and use AWS Glue's built-in data lineage tracking feature.
- B
Use Amazon S3 versioning to maintain historical versions of the data and track all changes made to the data.
- C
Implement AWS CloudTrail to log all API calls related to data access and transformation within the pipeline.
- D
Leverage AWS Glue DataBrew profiling capabilities to generate insights about data quality and transformations.
Show answer and explanation
Correct answer: A
Explanation
To ensure the accuracy and trustworthiness of data in a pipeline, tracking data lineage is essential. AWS Glue Data Catalog includes features for metadata management and lineage tracking, enabling teams to audit data flow end-to-end, understand transformations, and ensure data integrity. Other options, such as S3 versioning and CloudTrail, provide partial solutions but do not fully address the need for lineage tracking and transformation auditing.
- A. Correct.
Correct. AWS Glue Data Catalog provides metadata management and data lineage tracking, which are crucial for understanding the origin of the data, the transformations applied, and ensuring trustworthiness in data pipelines.
- B. Incorrect.
Incorrect. While Amazon S3 versioning can help maintain historical versions of data, it does not provide detailed tracking of data transformations or lineage, which is essential for ensuring data trustworthiness.
- C. Incorrect.
Incorrect. AWS CloudTrail logs API calls but does not provide data lineage or transformation details. It is primarily used for auditing API-level activities and security monitoring.
- D. Incorrect.
Incorrect. AWS Glue DataBrew focuses on data profiling and quality improvements but does not offer end-to-end data lineage tracking to ensure accuracy and trustworthiness of the data.