DEA-C01 Question 344
Select 3You are designing a data pipeline using AWS Glue to process customer transaction data stored in Amazon S3. The data contains various inconsistencies such as missing values, duplicate rows, and mismatched data types. Which cleansing techniques should you apply in this scenario to ensure the data is accurate and usable for downstream analytics?
- A
Remove duplicate records using AWS Glue's built-in transformation functions.
- B
Fill missing values with default or statistically computed values using Apache Spark's DataFrame functions in AWS Glue.
- C
Manually correct mismatched data types by downloading the dataset and modifying it locally before re-uploading it to S3.
- D
Standardize data formats and enforce schema consistency using AWS Glue's DynamicFrame transformations.
- E
Ignore data inconsistencies as they may not significantly impact downstream analytics.
Show answer and explanation
Correct answers: A, B, D
Explanation
Inconsistent, incomplete, or duplicate data can compromise the accuracy of analytics and insights. AWS Glue offers scalable and efficient tools for cleansing data, such as removing duplicates, handling missing values, and standardizing data formats, which are essential steps in ensuring data quality. Manual or ad-hoc approaches are not recommended in cloud-based pipelines due to inefficiency and scalability concerns, and ignoring issues can lead to poor results.
- A. Correct.
Correct. Removing duplicate records ensures that the data is not skewed by repeated entries, which is a standard cleansing technique.
- B. Correct.
Correct. Filling missing values with defaults or statistically computed values ensures completeness and improves the quality of the data for analytics.
- C. Incorrect.
Incorrect. Manually correcting mismatched data types locally is not efficient or scalable in a cloud-based data pipeline. AWS Glue provides built-in tools for handling such issues.
- D. Correct.
Correct. Standardizing data formats and enforcing schema consistency ensures that the data conforms to expected structures, reducing errors in downstream processing.
- E. Incorrect.
Incorrect. Ignoring data inconsistencies can significantly impact the accuracy and reliability of downstream analytics, making this approach unsuitable.