Databricks Data Engineer Professional Question 302
Select 3You are tasked with deploying a Databricks job using the Databricks CLI. The job runs a data pipeline on a daily schedule. Once deployed, you also need to monitor the job's run status and retrieve logs if the job fails. Which of the following CLI commands should you use for these tasks?
- A
databricks jobs create --json-file job-config.json
- B
databricks jobs run-now --job-id
- C
databricks jobs list
- D
databricks runs get --run-id
- E
databricks clusters restart --cluster-id
- F
databricks jobs delete --job-id
Show answer and explanation
Correct answers: A, C, D
Explanation
To deploy and monitor a Databricks job using the CLI, you first use the 'databricks jobs create' command to deploy the job. For monitoring, the 'databricks jobs list' command helps you identify the job and its runs, while 'databricks runs get' retrieves the status and logs for specific runs. The other options either perform unrelated tasks or are not relevant to this specific scenario.
- A. Correct.
This command is used to create a new job in Databricks using a job configuration file. It is required to initially deploy the job.
- B. Incorrect.
This command is used to manually trigger a job run. However, it is not required in this scenario because the job is scheduled to run daily.
- C. Correct.
This command lists all jobs in the workspace, which is helpful for monitoring and identifying job IDs.
- D. Correct.
This command retrieves details about a specific job run, including its status and logs, which is critical for monitoring and debugging.
- E. Incorrect.
This command is used to restart a cluster but is not relevant for deploying or monitoring Databricks jobs.
- F. Incorrect.
This command is used to delete a Databricks job, which is not relevant to the given scenario.