Databricks Data Engineer Professional Question 194
Single answerYou are tasked with designing a data pipeline in Databricks to handle Slowly Changing Dimensions (SCD) for a customer table. The table should track historical changes to customer addresses while ensuring that minor updates, such as phone number corrections, overwrite the current record without retaining history. Which SCD implementation approach should you choose?
- A
SCD Type 0
- B
SCD Type 1
- C
SCD Type 2
- D
A custom hybrid of SCD Type 1 and Type 2
Show answer and explanation
Correct answer: C
Explanation
SCD Type 2 is the correct choice because it supports tracking historical changes by creating new records for updates. At the same time, it enables overwriting the current record for minor updates, such as phone number corrections. This approach meets both key requirements in the scenario.
- A. Incorrect.
SCD Type 0 does not allow for any changes to the data, as it enforces immutability. This would not satisfy the requirement to update the current record or track historical changes.
- B. Incorrect.
SCD Type 1 overwrites the current record without retaining history. While this satisfies the phone number correction requirement, it does not track historical changes, which is a key requirement.
- C. Correct.
SCD Type 2 tracks historical changes by creating new records for each update while maintaining the ability to overwrite minor updates to the current record. This satisfies both requirements in the scenario.
- D. Incorrect.
A custom hybrid implementation might technically work, but it is unnecessary because SCD Type 2 already fulfills the requirements as described in the scenario.