Databricks Machine Learning Professional Question 110
Select 2You are tasked with designing a Databricks workflow where a notification must be sent to an external monitoring system every time a job completes or fails. The external system uses a Webhook to integrate with external tools. How can you achieve this functionality in Databricks?
- A
Configure a Webhook notification in the Databricks Jobs UI under the 'Advanced options' section.
- B
Create a Job in Databricks and configure a Webhook notification in the 'Job notification' settings.
- C
Use the Databricks REST API to configure a Webhook URL for the specific Job's notification settings.
- D
Manually call the Webhook URL using a separate Python script scheduled to run after the Job completes.
- E
Configure a Webhook listener in the Databricks cluster configuration settings.
Show answer and explanation
Correct answers: B, C
Explanation
Databricks supports Webhook integration for Job notifications through the 'Job notification' settings in the Databricks Jobs interface or via the Databricks REST API. These methods allow you to specify a Webhook URL to receive notifications based on Job events like completion or failure. Alternative methods, like manually calling a Webhook via a script or configuring Webhooks in cluster settings, are either inefficient or not supported.
- A. Incorrect.
Incorrect: While the Databricks Jobs UI does have an 'Advanced options' section, it does not provide a direct way to configure Webhook notifications. Notifications must be set up in the 'Job notification' settings or via API.
- B. Correct.
Correct: The 'Job notification' settings in Databricks Jobs allow you to configure a Webhook URL to send notifications upon job completion, failure, or partial completion.
- C. Correct.
Correct: The Databricks REST API supports configuring Job notifications, including specifying a Webhook URL for event-based notifications.
- D. Incorrect.
Incorrect: While you can manually call a Webhook using a custom script, this approach is not efficient or recommended when Databricks provides built-in support for Webhooks in its Job notification settings.
- E. Incorrect.
Incorrect: Webhook listeners cannot be configured in the cluster configuration settings. Cluster settings are used for cluster-level settings and not for Job notifications.