DEA-C01 Question 408
Single answerYou are working as a Data Engineer for a retail company. The company has a large dataset stored in Amazon S3 containing customer purchase data. You need to understand the structure, quality, and statistical distribution of the data to ensure it's ready for downstream analytics. Which AWS service and technique should you use to perform data profiling efficiently?
- A
Use AWS Glue DataBrew to generate data quality statistics and identify anomalies in the dataset.
- B
Use Amazon Quicksight to create dashboards for visualizing the dataset.
- C
Use Amazon Athena to query the data and manually inspect the results.
- D
Use AWS Lambda to write custom Python scripts for processing and profiling the dataset.
Show answer and explanation
Correct answer: A
Explanation
Data profiling involves understanding the structure, quality, and distribution of data. AWS Glue DataBrew is a managed service designed for data profiling and preparation, offering automated tools to generate insights into the dataset's quality and structure. This makes it the most efficient and appropriate choice compared to other options.
- A. Correct.
This is the correct option. AWS Glue DataBrew is specifically designed to help users profile, clean, and transform data efficiently. It can generate data quality statistics such as missing values, outliers, and data distributions, making it ideal for data profiling tasks.
- B. Incorrect.
This is incorrect. While Amazon Quicksight is a powerful tool for data visualization, it is not designed for detailed data profiling or generating statistical summaries of datasets.
- C. Incorrect.
This is incorrect. Although Amazon Athena allows querying of datasets, it requires manual querying and inspection, which is less efficient and not specifically tailored for data profiling.
- D. Incorrect.
This is incorrect. While AWS Lambda can run custom scripts for profiling, it requires significant development effort and lacks built-in features for data profiling compared to a service like AWS Glue DataBrew.