Databricks Generative AI Engineer Associate Question 64
Select 4You are tasked with processing a large text dataset that has been chunked into smaller parts. The goal is to write this chunked text into a Delta Lake table registered in Unity Catalog. Which of the following steps should you include in your workflow to achieve this?
- A
Use the
createOrReplace()method to define the Delta table in Unity Catalog before writing data. - B
Enable schema enforcement and write each chunk as a DataFrame to the Delta table using the
appendmode. - C
Partition the Delta table by a meaningful column (e.g., timestamp or category) to optimize performance.
- D
Register the Delta table in Unity Catalog by specifying the Unity Catalog database and table name in the write operation.
- E
Ensure that Unity Catalog permissions are properly configured for all users and services involved in the workflow.
Show answer and explanation
Correct answers: B, C, D, E
Explanation
To write chunked text into a Delta Lake table in Unity Catalog, you need to enable schema enforcement, use append mode for incremental writes, optimize performance by partitioning the table, and ensure the table is registered in Unity Catalog using the database and table name. Additionally, proper permission configurations in Unity Catalog are essential for secure access and management. The createOrReplace() method, however, is not relevant for Unity Catalog table creation.
- A. Incorrect.
The
createOrReplace()method is not used to define tables in Unity Catalog. Unity Catalog tables are typically created using SQL or other table creation methods. - B. Correct.
Enabling schema enforcement and writing data in
appendmode ensures that the chunked text is written to the Delta table without overwriting existing data, while maintaining schema consistency. - C. Correct.
Partitioning the Delta table by a meaningful column improves performance for queries and data retrieval as Delta Lake optimizes access to partitioned data.
- D. Correct.
Registering the Delta table in Unity Catalog by specifying the database and table name ensures the table is discoverable and accessible via Unity Catalog.
- E. Correct.
Configuring Unity Catalog permissions ensures secure access to the Delta table and is a critical step for operationalizing workflows in a multi-user environment.