DEA-C01 Question 285
Single answerYou are managing a data lake on Amazon S3 using AWS Glue for ETL jobs and Amazon Athena for querying the data. A new requirement arises to add a new column to your dataset without breaking existing queries and ETL jobs. Which schema evolution technique would be most appropriate to handle this change?
- A
Add the new column to the schema and make it a required field in the AWS Glue Data Catalog.
- B
Add the new column to the schema and make it an optional field in the AWS Glue Data Catalog.
- C
Drop the existing table in the AWS Glue Data Catalog and recreate it with the new column.
- D
Create a new table in the AWS Glue Data Catalog to accommodate the new schema and migrate the data.
Show answer and explanation
Correct answer: B
Explanation
Schema evolution techniques aim to handle schema changes in a way that does not disrupt existing workflows. Adding a new column as an optional field in the AWS Glue Data Catalog ensures backward compatibility, as existing queries and ETL jobs can continue to function without requiring data for the new column. This approach minimizes disruption while accommodating the schema change.
- A. Incorrect.
Marking the new column as required would break existing ETL jobs and queries that do not provide data for the new column, leading to failures.
- B. Correct.
Making the new column optional ensures backward compatibility, allowing queries and ETL jobs to run without requiring data for the new column.
- C. Incorrect.
Dropping and recreating the table is not recommended as it would disrupt existing queries and potentially lead to data loss or operational downtime.
- D. Incorrect.
Creating a new table requires additional operational overhead and migration efforts, which are unnecessary for this use case.