DEA-C01 Question 202
Select 3You are a data engineer tasked with creating a data catalog for an S3-based data lake. The catalog should enable querying the data using Amazon Athena. Which steps should you take to create and populate the data catalog effectively?
- A
Use AWS Glue to create a crawler and configure it to scan the S3 bucket containing the data.
- B
Manually create tables in the AWS Glue Data Catalog by specifying the schema and location of the S3 data.
- C
Run the AWS Glue crawler to automatically detect the schema and populate the AWS Glue Data Catalog.
- D
Ensure that the S3 bucket has a policy allowing access to AWS Glue and Athena services.
- E
Launch an Amazon RDS database to store metadata for the data catalog.
Show answer and explanation
Correct answers: A, C, D
Explanation
To create a data catalog for an S3-based data lake, AWS Glue crawlers are typically used to scan the S3 bucket, detect schemas, and populate the Glue Data Catalog. This catalog serves as a metadata repository that is natively integrated with Amazon Athena, enabling efficient querying. It's also critical to ensure that the S3 bucket permissions allow Glue and Athena to access the data. While manual table creation and external databases are possible, they are not necessary for this use case and would complicate the workflow.
- A. Correct.
Correct. AWS Glue crawlers are designed to scan data sources like S3 and automatically detect the schema, creating tables in the Glue Data Catalog.
- B. Incorrect.
Incorrect. While you can manually create tables in AWS Glue, this is not the recommended approach for automating and managing schema detection, especially when dealing with large data lakes.
- C. Correct.
Correct. Running the AWS Glue crawler is essential to automatically populate the Glue Data Catalog with the metadata and schema of your S3 data.
- D. Correct.
Correct. To enable AWS Glue and Athena to access the S3 bucket, proper permissions must be granted via an S3 bucket policy.
- E. Incorrect.
Incorrect. AWS Glue Data Catalog serves as the metadata repository, and there is no need to launch an Amazon RDS database for this purpose.