Databricks Machine Learning Professional Question 135
Select 3You are tasked with deploying a machine learning model in a batch inference pipeline on Databricks. The goal is to compute predictions on a large dataset and store the results in a Delta table for downstream analysis. Which of the following steps are required to implement this batch deployment effectively?
- A
Load the input data into a Spark DataFrame and apply the model's
predictmethod to generate predictions. - B
Save the predictions to a Delta table for downstream analysis.
- C
Use Databricks Jobs to schedule and automate the batch inference pipeline.
- D
Deploy the model as an online REST API endpoint using Databricks Model Serving.
- E
Manually partition the data into small chunks and process each chunk sequentially outside of Spark.
Show answer and explanation
Correct answers: A, B, C
Explanation
Batch deployment in Databricks involves loading data into a Spark DataFrame, applying the model to generate predictions, and storing the results in a Delta table. Automating and scheduling the pipeline with Databricks Jobs ensures the process runs efficiently at regular intervals. Using a REST API or manually partitioning the data are not appropriate for this use case, as they are designed for real-time serving or inefficient processing, respectively.
- A. Correct.
Correct. In a batch deployment, the input data is typically loaded into a Spark DataFrame, and predictions are computed using the model's
predictmethod or equivalent. This allows for efficient processing of large datasets. - B. Correct.
Correct. After computing predictions, saving them to a Delta table is a common practice for storing results in a scalable and queryable format.
- C. Correct.
Correct. Databricks Jobs can be used to automate and schedule the batch inference pipeline, ensuring predictions are computed regularly without manual intervention.
- D. Incorrect.
Incorrect. Deploying the model as an online REST API endpoint is designed for real-time serving, not batch inference. Batch deployment does not involve a REST API.
- E. Incorrect.
Incorrect. Manually partitioning the data and processing it outside of Spark is inefficient and defeats the purpose of using Spark's distributed computing capabilities for batch inference.