DEA-C01 Question 266
Select 2You are a data engineer at a company that uses AWS Glue for ETL processing and Amazon Redshift for analytics. Your team is concerned about the accuracy and trustworthiness of the data being ingested into Redshift. You decide to implement a solution to track the data lineage throughout the ETL process. Which of the following actions will help you ensure data lineage and thereby improve data accuracy and trustworthiness?
- A
Enable AWS Glue job bookmarks to track processed data.
- B
Use AWS Glue's built-in data lineage tracking feature to visualize the flow of data transformations.
- C
Implement Amazon Redshift's audit logging to monitor user queries and table modifications.
- D
Tag data sources, transformations, and outputs with metadata describing their origins and transformations.
- E
Integrate AWS Glue with AWS Lake Formation to track permissions and data access patterns.
Show answer and explanation
Correct answers: B, D
Explanation
Ensuring data lineage involves understanding the origins and transformations of data across the pipeline. AWS Glue's built-in lineage tracking and tagging metadata are effective ways to ensure accurate tracking of data transformations, improving trustworthiness and accuracy. Other options, like job bookmarks or audit logging, serve different purposes and do not directly address data lineage.
- A. Incorrect.
AWS Glue job bookmarks track the state of data processing to prevent reprocessing but do not provide detailed lineage information about transformations or origins.
- B. Correct.
AWS Glue has built-in data lineage tracking features that allow you to visualize the flow of data transformations, helping ensure data accuracy and trustworthiness.
- C. Incorrect.
Amazon Redshift's audit logging is useful for monitoring user activity and table changes but does not provide insights into data lineage.
- D. Correct.
Tagging data sources, transformations, and outputs with metadata allows you to document and track the origins and transformations of data, which is key to ensuring data lineage.
- E. Incorrect.
While AWS Lake Formation helps manage permissions and access patterns, it does not provide detailed data lineage tracking for transformations or origins.