Databricks Machine Learning Professional Question 125
Select 3You are tasked with deploying a machine learning model to perform batch inference for a large dataset stored in a Delta table on Databricks. The model is a trained scikit-learn model that predicts customer churn. The batch inference results should be written back to another Delta table for downstream analytics. Which of the following steps are required to successfully deploy and execute batch inference in this scenario?
- A
Load the trained model using MLflow and apply it to a Spark DataFrame created from the Delta table.
- B
Use Spark UDFs to distribute the model inference logic across the worker nodes.
- C
Export the Delta table to a local CSV file and run the inference on a standalone Python environment.
- D
Write the inference results to a new Delta table using Spark's write API.
- E
Deploy the model as a REST API endpoint for real-time predictions.
Show answer and explanation
Correct answers: A, B, D
Explanation
To perform batch inference in Databricks, you need to load the model (e.g., using MLflow), apply it to a Spark DataFrame created from the Delta table, and distribute the inference computation using Spark UDFs. Finally, the results should be written back to a Delta table for further use. Exporting the Delta table to a standalone environment or deploying the model as a REST API endpoint is not appropriate for this batch scenario.
- A. Correct.
This is correct because MLflow can be used to load the trained scikit-learn model, and Spark DataFrames are required for processing data stored in Delta tables.
- B. Correct.
This is correct because using Spark UDFs allows the inference logic to be distributed across the Spark cluster, enabling scalable batch processing.
- C. Incorrect.
This is incorrect because exporting the Delta table to a local CSV file and running inference in a standalone Python environment is inefficient and does not leverage the distributed computing capabilities of Databricks.
- D. Correct.
This is correct because the results of batch inference need to be written back to a Delta table for downstream analytics, and Spark's write API supports this operation.
- E. Incorrect.
This is incorrect because deploying the model as a REST API endpoint is typically used for real-time predictions, not batch inference.