Databricks Machine Learning Professional Question 184
Single answerYou are tasked with deploying a machine learning model for continuous time-based predictions in a streaming environment. The model needs to predict energy consumption for the next hour based on real-time sensor data arriving every minute. Which of the following approaches is most suitable for ensuring accurate and efficient real-time predictions in this scenario?
- A
Use a batch inference pipeline that processes sensor data every hour.
- B
Deploy the model with a structured streaming pipeline to process data and make predictions as new data arrives.
- C
Manually collect data in real-time, store it in a database, and run predictions once the database has sufficient data.
- D
Use a static pre-trained model without updates and run predictions periodically on historical data.
Show answer and explanation
Correct answer: B
Explanation
Structured streaming is specifically designed for continuous, real-time data processing and is ideal for time-based predictions in streaming deployments. In this case, it ensures that the model processes sensor data as it arrives and makes predictions in near real-time, which aligns with the requirements of the scenario.
- A. Incorrect.
Batch inference pipelines are not optimal for real-time streaming scenarios because they process data in large chunks, leading to delays and are unsuitable for minute-by-minute predictions.
- B. Correct.
Deploying the model with a structured streaming pipeline allows for continuous ingestion, processing, and prediction as the data arrives, making it the most suitable approach for real-time predictions.
- C. Incorrect.
Manually collecting real-time data and relying on database queries for prediction introduces latency and complexity, making it an inefficient method for a streaming deployment.
- D. Incorrect.
Using a static pre-trained model without updates may work for periodic predictions but is not suitable for continuous real-time predictions, as it fails to adapt to new data in a streaming environment.