DEA-C01 Question 195
Select 3A data engineering team is building a data lake on Amazon S3 and wants to enable users to easily discover and query the data. They plan to use AWS Glue Data Catalog for this purpose. Which of the following steps should they take to ensure the data is properly cataloged and queryable?
- A
Use AWS Glue Crawlers to automatically discover and register metadata for the datasets in the Glue Data Catalog.
- B
Manually create AWS Glue tables for each dataset in the Glue Data Catalog.
- C
Ensure the IAM role used by the AWS Glue Crawler has sufficient permissions to access the S3 bucket and objects.
- D
Use Amazon Redshift to directly catalog the data in S3 without involving AWS Glue.
- E
Enable integration between AWS Glue Data Catalog and Amazon Athena for querying the data.
Show answer and explanation
Correct answers: A, C, E
Explanation
AWS Glue Data Catalog serves as a central metadata repository for datasets in AWS. To catalog data in a data lake, AWS Glue Crawlers can be used to automatically discover and register metadata. It is essential to ensure the crawler has the correct IAM permissions to access S3 data. Additionally, enabling integration with Amazon Athena allows users to query the cataloged data directly from S3, making the data both discoverable and queryable. Manual table creation and direct use of Amazon Redshift are not suitable approaches for this scenario.
- A. Correct.
Correct: AWS Glue Crawlers are designed to automatically discover and extract metadata (like schema and data types) and register it in the Glue Data Catalog, making this a key step for cataloging data.
- B. Incorrect.
Incorrect: While manually creating tables is an option, it is not efficient or scalable for large data lakes. AWS Glue Crawlers provide an automated approach for this task.
- C. Correct.
Correct: The IAM role used by the AWS Glue Crawler must have sufficient permissions to access the S3 buckets and objects to properly catalog the data.
- D. Incorrect.
Incorrect: Amazon Redshift is a data warehouse service and does not directly catalog data in S3. AWS Glue Data Catalog is the appropriate service for this use case.
- E. Correct.
Correct: Amazon Athena relies on the AWS Glue Data Catalog to query structured data directly from S3. Enabling this integration ensures that the cataloged data can be queried efficiently.