Databricks Data Engineer Professional Question 103
Select 3You are working on a Delta Lake table that tracks customer orders. The table is frequently updated, and sometimes rows are deleted. You need to propagate these deletes to a downstream system that consumes this table using Change Data Feed (CDF). Which of the following steps are required to ensure that deletes are correctly propagated to the downstream system?
- A
Enable Change Data Feed (CDF) on the Delta table using table properties.
- B
Query the
_change_typecolumn in the Change Data Feed to identify deleted rows. - C
Configure the downstream system to accept updates but ignore deletes.
- D
When reading the Change Data Feed, filter rows where
_change_typeis 'delete'. - E
Manually generate delete events in the downstream system based on the Delta table state.
Show answer and explanation
Correct answers: A, B, D
Explanation
To propagate deletes using Change Data Feed (CDF), you must first enable CDF on the Delta table. Then, by querying the _change_type column, you can identify deleted rows. You must explicitly filter for rows where _change_type is 'delete' to ensure those deletions are handled correctly in the downstream system. CDF automates the process of capturing changes, so manual generation of delete events is unnecessary.
- A. Correct.
This is correct. Change Data Feed must be enabled on the Delta table to capture row-level changes, including deletes.
- B. Correct.
This is correct. The
_change_typecolumn in the Change Data Feed helps identify the type of change, such as 'insert', 'update', or 'delete'. - C. Incorrect.
This is incorrect. To propagate deletes, the downstream system must be configured to process deletes rather than ignoring them.
- D. Correct.
This is correct. Filtering for rows where
_change_typeis 'delete' ensures that the deleted rows are correctly identified and propagated to the downstream system. - E. Incorrect.
This is incorrect. Manually generating delete events is not required when using Change Data Feed, as CDF automatically captures and propagates delete operations.