Databricks Machine Learning Professional Question 193
Select 3You are tasked with converting a batch machine learning deployment pipeline to a streaming deployment pipeline in Databricks for a model predicting customer churn. The batch pipeline uses a Delta table as the data source and writes results to another Delta table. Which of the following steps are necessary to enable a streaming deployment pipeline in Databricks?
- A
Change the data source to read from a Delta table in streaming mode using
.readStream. - B
Update the write operation to use
.write()instead of.writeStream(). - C
Ensure that the model inference logic can handle micro-batches.
- D
Configure a checkpoint location in the streaming write operation.
- E
Use Spark Structured Streaming's
trigger(once=True)mode to simulate streaming.
Show answer and explanation
Correct answers: A, C, D
Explanation
To convert a batch deployment pipeline to a streaming deployment pipeline in Databricks, the data source must be read in streaming mode using .readStream(), the model inference logic must support micro-batches, and a checkpoint location should be configured to ensure state management and fault tolerance. Using .write() or trigger(once=True) does not create a true streaming pipeline.
- A. Correct.
Correct: To enable streaming, the data source must be read in streaming mode using
.readStreaminstead of.read. - B. Incorrect.
Incorrect: The write operation must use
.writeStream()for streaming pipelines, not.write(). - C. Correct.
Correct: Model inference logic must be designed to process micro-batches, as streaming involves processing data incrementally.
- D. Correct.
Correct: Checkpointing is crucial in streaming pipelines to maintain state and ensure fault tolerance.
- E. Incorrect.
Incorrect: While
trigger(once=True)processes data in a batch-like manner, it does not create a continuous streaming pipeline.