Databricks Data Engineer Professional Question 283
Single answerYou are tasked with creating a multi-task job in Databricks to process customer orders. The workflow should follow these steps: 1) Load raw data into a Delta table, 2) Perform data transformation, and 3) Generate a summary report. The data transformation step should only start after the data loading step is complete, and the summary report step should only start after the transformation step is complete. Which configuration ensures the correct dependencies and execution order in the multi-task job?
- A
Configure the job with three tasks: Task 1 for data loading, Task 2 for data transformation, and Task 3 for summary report. Set Task 2 to depend on Task 1, and Task 3 to depend on Task 2.
- B
Create a single task in the job and write a single notebook to handle all three steps sequentially.
- C
Configure the job with three tasks: Task 1 for data loading, Task 2 for data transformation, and Task 3 for summary report. Set Task 2 to depend on Task 3, and Task 3 to depend on Task 1.
- D
Configure the job with three tasks: Task 1 for data loading, Task 2 for data transformation, and Task 3 for summary report. Set Task 1 to depend on Task 2, and Task 3 to depend on Task 2.
Show answer and explanation
Correct answer: A
Explanation
In Databricks, multi-task jobs allow you to define dependencies between tasks, ensuring that tasks execute in the desired order. To satisfy the requirements, Task 2 should depend on Task 1, and Task 3 should depend on Task 2. This ensures a clear and logical flow of execution: data loading -> data transformation -> summary report.
- A. Correct.
Correct: This configuration ensures the correct dependency order. Task 2 starts only after Task 1, and Task 3 starts only after Task 2, aligning with the requirements.
- B. Incorrect.
Incorrect: Using a single task for all steps does not leverage the multi-task job capabilities, making it harder to manage and debug.
- C. Incorrect.
Incorrect: This dependency configuration creates a circular dependency (Task 2 depends on Task 3, and Task 3 depends on Task 1), which is invalid.
- D. Incorrect.
Incorrect: This configuration misorders the dependencies, as Task 1 depends on Task 2, which contradicts the required sequence.