Databricks Machine Learning Professional Question 119
Single answerYou are designing a machine learning pipeline in Databricks and want to trigger a real-time notification to a monitoring system whenever a model training job fails. You decide to use an HTTP webhook for this purpose. Where should the Webhook URL come from, and what is an appropriate use case for HTTP webhooks in this scenario?
- A
The Webhook URL should come from the monitoring system, and HTTP webhooks are used to enable real-time event notifications.
- B
The Webhook URL should come from the Databricks workspace, and HTTP webhooks are used to log all events asynchronously.
- C
The Webhook URL should be dynamically generated during each model training run, and HTTP webhooks are used to store metadata about training jobs.
- D
The Webhook URL should come from the Databricks REST API, and HTTP webhooks are used for scheduling future jobs.
Show answer and explanation
Correct answer: A
Explanation
HTTP webhooks are a mechanism to send real-time notifications to external systems when specific events occur. In this case, when a model training job fails, the Webhook URL provided by the monitoring system can be used to send notifications to the system for immediate action. This approach ensures near-instantaneous communication and is a common use case for webhooks.
- A. Correct.
This is the correct answer. The Webhook URL is typically provided by the external monitoring system or service, and HTTP webhooks are well-suited for real-time event notifications, such as alerting on job failures.
- B. Incorrect.
Incorrect. The Webhook URL does not originate from the Databricks workspace. HTTP webhooks are not primarily used for logging all events asynchronously; other tools like MLflow Tracking or Databricks Jobs logs are better suited for that.
- C. Incorrect.
Incorrect. The Webhook URL is not dynamically generated during each model training run. HTTP webhooks are not designed to store metadata; instead, they are used to notify external systems of events.
- D. Incorrect.
Incorrect. The Webhook URL does not come from the Databricks REST API, and HTTP webhooks are not used for scheduling jobs. REST APIs are used for direct interaction and control of Databricks resources.