Databricks Data Engineer Associate Question 433
Select 2You are designing a Databricks workflow to process a large dataset. The process involves three distinct tasks: Task A performs data ingestion and writes raw data to a Delta table, Task B cleanses the data from the Delta table and writes the results to another Delta table, and Task C generates summary reports based on the cleansed data. How should you configure the workflow to ensure correct execution?
- A
Set Task A as a predecessor for Task B.
- B
Set Task B as a predecessor for Task C.
- C
Allow Task C to run in parallel with Task A.
- D
Allow Task B to run in parallel with Task A.
- E
Set Task C as a predecessor for Task B.
Show answer and explanation
Correct answers: A, B
Explanation
In a Databricks workflow, tasks should be configured with dependencies to ensure data is processed in the correct order. Task A must finish ingestion before Task B can cleanse the data, and Task B must finish cleansing before Task C can generate reports. Setting up Task A as a predecessor for Task B and Task B as a predecessor for Task C ensures the workflow executes in the correct sequence.
- A. Correct.
Task A must complete before Task B can start because Task B relies on the output Delta table created by Task A.
- B. Correct.
Task B must complete before Task C can start because Task C relies on the cleansed data written by Task B.
- C. Incorrect.
Task C cannot run in parallel with Task A because Task C depends on the cleansed data from Task B, which in turn depends on Task A.
- D. Incorrect.
Task B cannot run in parallel with Task A because Task B relies on the Delta table created by Task A.
- E. Incorrect.
Task C cannot be a predecessor for Task B because Task B processes data before Task C generates reports.