Databricks Machine Learning Professional Question 172
Single answerA data engineering team has developed a machine learning model to predict customer churn based on incoming customer usage data. The data arrives in real-time as a continuous stream. The team wants to deploy the model to provide predictions in near real-time and integrate the output into a dashboard. Which solution best addresses this requirement?
- A
Use Databricks Structured Streaming to process the incoming data and apply the model for real-time predictions.
- B
Use a batch processing job in Databricks to periodically process the incoming data and apply the model.
- C
Deploy the model as a REST API and use an external application to fetch the streaming data and make predictions.
- D
Use a Databricks notebook to manually trigger the model for predictions whenever new data arrives.
Show answer and explanation
Correct answer: A
Explanation
Databricks Structured Streaming is specifically designed to handle continuous data streams and enables real-time inference by applying machine learning models on incoming data. This makes it the most appropriate solution for the described scenario, where predictions need to be made in near real-time and integrated into a dashboard. Other options either do not meet the real-time requirement or lack scalability and efficiency.
- A. Correct.
This is the correct solution because Databricks Structured Streaming supports continuous data processing and can apply machine learning models in real-time, making it suitable for continuous inference.
- B. Incorrect.
Batch processing is not suitable for real-time or near real-time requirements as it processes data in discrete intervals rather than continuously.
- C. Incorrect.
While deploying a REST API for the model is possible, it does not leverage Databricks Structured Streaming and would require additional infrastructure for fetching and processing streaming data.
- D. Incorrect.
Manually triggering the model in a notebook is inefficient and impractical for real-time predictions on continuous data streams.