Databricks Data Engineer Professional Question 310
Single answerYou are tasked with automating a data pipeline in Databricks. As part of the workflow, you need to clone an existing job, trigger a run of the cloned job with specific parameters, and export the run output for further analysis. Which sequence of Databricks REST API operations should you use to accomplish this?
- A
Use the 'Jobs Clone' API to create a new job, then the 'Jobs Run Now' API to trigger the run, and finally the 'Runs Export' API to export the run output.
- B
Use the 'Jobs Create' API to create a new job, then the 'Jobs Trigger Run' API to trigger the run, and finally the 'Runs Get Output' API to export the run output.
- C
Use the 'Jobs Update' API to modify the job, then the 'Jobs Run Submit' API to trigger the run, and finally the 'Runs Export' API to export the run output.
- D
Use the 'Jobs Clone' API to create a new job, then the 'Jobs Run Submit' API to trigger the run, and finally the 'Runs Get Output' API to export the run output.
Show answer and explanation
Correct answer: A
Explanation
To achieve the required functionality, the correct sequence of REST API operations involves using the 'Jobs Clone' API to duplicate an existing job, 'Jobs Run Now' API to trigger the run of the cloned job, and 'Runs Export' API to retrieve the output of the run. This workflow leverages the appropriate APIs for cloning, running, and exporting job outputs as specified in Databricks REST API documentation.
- A. Correct.
Correct. The 'Jobs Clone' API duplicates an existing job, 'Jobs Run Now' triggers the run of the cloned job, and 'Runs Export' retrieves the output of the run.
- B. Incorrect.
Incorrect. While 'Jobs Create' can be used to create a new job, the question specifies cloning an existing job, which requires the 'Jobs Clone' API. Additionally, there is no 'Jobs Trigger Run' API.
- C. Incorrect.
Incorrect. The 'Jobs Update' API is for modifying existing jobs, not cloning them. Furthermore, 'Jobs Run Submit' is used for one-time runs without saving them as a job, which does not align with the question.
- D. Incorrect.
Incorrect. While 'Jobs Clone' can be used to duplicate a job, 'Jobs Run Submit' is for one-time runs and does not trigger a saved job. Additionally, the 'Runs Get Output' API is used to fetch logs and output files but does not directly export run output.