Databricks Data Engineer Professional Question 179
Single answerYou are working on a Delta Lake table in Databricks that stores customer transaction data. To ensure data quality, you want to enforce a constraint that prevents rows with negative transaction amounts from being written to the table. Which of the following approaches should you take to implement this requirement?
- A
Use Delta Lake's CHECK constraint to enforce the condition during writes.
- B
Use a Databricks notebook to manually validate data before writing to the Delta table.
- C
Enable Delta Lake's Auto Optimize feature to automatically detect and prevent bad data.
- D
Create a Delta table with a NOT NULL constraint on the transaction amount column.
Show answer and explanation
Correct answer: A
Explanation
Delta Lake's CHECK constraints allow you to define rules for data integrity directly within the table schema. By using a CHECK constraint, you can enforce that only valid data (in this case, non-negative transaction amounts) is written to the Delta table. This is an automated, reliable, and scalable way to maintain data quality.
- A. Correct.
Delta Lake's CHECK constraint is specifically designed to enforce conditions on data being written to a table, such as ensuring values in a column meet a certain condition (e.g., transaction_amount >= 0). This is the correct approach for the scenario.
- B. Incorrect.
Manually validating data in a Databricks notebook is error-prone and not a scalable or automated solution to enforce constraints. This option does not directly address the problem.
- C. Incorrect.
Delta Lake's Auto Optimize feature improves table performance and compaction but does not enforce data quality constraints or prevent bad data from being written.
- D. Incorrect.
A NOT NULL constraint ensures that a column cannot have NULL values, but it does not prevent invalid values, such as negative numbers, from being written to the table.