Databricks Data Engineer Professional Question 71
Select 3You are working on a Spark table in Delta Lake that stores customer records. The table uses Type 1 updates, where only the latest version of each record is maintained. You need to update multiple customer records based on a set of changes provided in a separate DataFrame. Which of the following strategies can you use to update the records in the table?
- A
Use the MERGE INTO statement with the appropriate conditions to match and update the records.
- B
Perform an overwrite operation by writing a new version of the table with updated data.
- C
Use a DELETE operation to remove the old records, followed by an INSERT to add the updated records.
- D
Utilize the UPDATE statement to directly modify the specific rows in the table.
- E
Apply a UNION operation to combine the existing records with the new updates and overwrite the table.
Show answer and explanation
Correct answers: A, B, D
Explanation
In a Type 1 update scenario for a Spark table, the primary goal is to replace or modify the existing records with the updated ones while maintaining only the latest data. MERGE INTO, table overwrites, and UPDATE are all valid strategies to achieve this in Delta Lake. DELETE followed by INSERT is not efficient and UNION is not appropriate for updating specific records in a table.
- A. Correct.
Correct: The MERGE INTO statement is a standard approach in Delta Lake for handling updates. It allows you to specify update conditions and is efficient for Type 1 updates.
- B. Correct.
Correct: Overwriting the table with a new version containing the updates is a valid strategy for Type 1 updates, where only the latest records are maintained.
- C. Incorrect.
Incorrect: Using DELETE followed by INSERT is not efficient or recommended for Type 1 updates, as it involves multiple operations and lacks transactional consistency.
- D. Correct.
Correct: The UPDATE statement in Delta Lake can be used to directly modify specific rows in the table, making it valid for Type 1 updates.
- E. Incorrect.
Incorrect: While UNION can combine datasets, it is not a valid or efficient approach for updating records in a Delta table.