Databricks Data Engineer Professional Question 301
Select 3You are tasked with deploying and monitoring a Databricks job using the Databricks CLI. The job reads data from a Delta Lake table, performs transformations, and writes the output to another Delta table. Which sequence of Databricks CLI commands is required to deploy the job and monitor its execution?
- A
Use the 'databricks jobs create' command to create the job and then 'databricks jobs run-now' to trigger the job.
- B
Use the 'databricks clusters create' command to create a cluster and then 'databricks clusters attach-job' to attach the job to the cluster.
- C
Monitor the job execution using the 'databricks jobs runs get' command to check the run status.
- D
Use the 'databricks fs cp' command to copy the job script to the Databricks workspace before deploying the job.
- E
Use the 'databricks jobs update' command to modify the job configuration after deployment.
Show answer and explanation
Correct answers: A, C, E
Explanation
To deploy and monitor a Databricks job using the CLI, the 'databricks jobs create' command is used for deployment, 'databricks jobs run-now' triggers the execution, and 'databricks jobs runs get' fetches the job run status. Additionally, the 'databricks jobs update' command allows modification of job configurations post-deployment. While 'databricks fs cp' can be useful for uploading scripts, it is not essential unless the job script needs to be transferred to Databricks.
- A. Correct.
Correct: The 'databricks jobs create' command is used to deploy a new job, and the 'databricks jobs run-now' command triggers the job execution immediately.
- B. Incorrect.
Incorrect: The 'databricks clusters create' command is used to create clusters, but clusters are not directly attached to jobs using 'databricks clusters attach-job'. Instead, clusters are specified in the job configuration.
- C. Correct.
Correct: The 'databricks jobs runs get' command is used to monitor the status of a job execution, such as checking if it has completed successfully or failed.
- D. Incorrect.
Incorrect: The 'databricks fs cp' command is used for file system operations, like copying files to or from DBFS, but it is not necessary for deploying a Databricks job unless the job script is being uploaded.
- E. Correct.
Correct: The 'databricks jobs update' command is used to modify the configuration of an existing job after it has been deployed, such as updating the job's schedule or parameters.