DEA-C01 Question 204
Select 2Your company wants to organize and manage metadata for an S3 data lake to make it searchable for analytics workloads. You must create a data catalog in AWS that integrates with AWS Glue for ETL and supports querying with Amazon Athena. What steps should you take to create and populate the data catalog effectively?
- A
Use AWS Glue to crawl the S3 bucket and populate the metadata in the Glue Data Catalog.
- B
Manually upload a JSON metadata file to the Glue Data Catalog for each dataset in the S3 bucket.
- C
Define a Glue database and use AWS Glue Crawlers to scan the data and update the catalog automatically.
- D
Use Amazon Redshift to query the S3 bucket and automatically populate the Glue Data Catalog.
- E
Enable automatic schema inference in Amazon Athena to directly populate the Glue Data Catalog without additional tools.
Show answer and explanation
Correct answers: A, C
Explanation
To create a data catalog for an S3 data lake, AWS Glue provides powerful tools like Glue Crawlers that can automatically scan S3 buckets, infer schemas, and populate metadata into the Glue Data Catalog. Defining a Glue database ensures that the metadata is organized correctly. Manual approaches or reliance on unrelated services like Redshift or Athena are either inefficient or not feasible for this task.
- A. Correct.
Correct. AWS Glue Crawlers can scan the S3 bucket, infer the schema, and populate the Glue Data Catalog automatically. This is a standard method for creating and maintaining a data catalog in AWS.
- B. Incorrect.
Incorrect. Manually uploading metadata is not an efficient or scalable solution, especially for large datasets. AWS Glue provides automated tools for this purpose.
- C. Correct.
Correct. Defining a Glue database and using Glue Crawlers ensures that the scanned metadata is organized and stored appropriately in the Glue Data Catalog.
- D. Incorrect.
Incorrect. Amazon Redshift is primarily a data warehousing service and does not automatically integrate with the Glue Data Catalog to populate metadata from S3. Redshift Spectrum can query S3, but it doesn't populate the Glue Data Catalog.
- E. Incorrect.
Incorrect. Amazon Athena uses the Glue Data Catalog for querying but does not have a feature to populate the catalog directly. Glue Crawlers are needed to scan and populate the catalog.