Databricks Data Engineer Professional Question 16
Select 3A data engineering team is using Delta Lake for a shared dataset with multiple users performing read and write operations concurrently. A new data engineer observes an error during their transaction, indicating a conflict. Which of the following scenarios could lead to a conflict in Delta Lake under Optimistic Concurrency Control?
- A
Two users attempt to update the same set of rows simultaneously.
- B
A user appends new rows to the same Delta table while another user reads the table.
- C
Two users delete overlapping rows in the same Delta table at the same time.
- D
A user overwrites the entire Delta table while another user is in the middle of reading it.
- E
Two users perform unrelated updates on different parts of the Delta table.
Show answer and explanation
Correct answers: A, C, D
Explanation
Delta Lake's Optimistic Concurrency Control ensures isolation by validating that no conflicting changes have occurred before committing a transaction. Conflicts arise when two transactions modify the same data or metadata, such as overlapping updates, deletes, or overwrites affecting the same data. Operations that do not overlap or only append data without altering existing rows typically do not conflict.
- A. Correct.
Correct: Delta Lake's Optimistic Concurrency Control detects conflicts when two users attempt to update the same rows, as it violates isolation rules.
- B. Incorrect.
Incorrect: Appending new rows to the table does not conflict with a read operation, as it does not modify existing data or metadata being read.
- C. Correct.
Correct: Deleting overlapping rows creates a conflict since both transactions modify the same part of the data, violating Delta Lake's isolation requirements.
- D. Correct.
Correct: Overwriting the entire table conflicts with a read operation because the metadata and data being read are altered by the overwrite.
- E. Incorrect.
Incorrect: Updates performed on different parts of the table do not conflict since they affect non-overlapping data and maintain isolation.