Databricks Machine Learning Associate Question 637
Select 3You are working on a machine learning project in Databricks. You have a trained scikit-learn model and a Pandas DataFrame named 'new_data' containing new instances for which predictions are required. Which of the following steps would allow you to perform batch inference using the model and the DataFrame efficiently?
- A
Use the model's predict() method on the DataFrame directly to generate predictions.
- B
Ensure that the feature columns in 'new_data' match the feature names used during training before calling the predict() method.
- C
Convert the Pandas DataFrame to a NumPy array before passing it to the model's predict() method.
- D
Use Spark DataFrame's
.toPandas()method to convert 'new_data' to a Pandas DataFrame before calling predict(). - E
Verify that there are no missing values in 'new_data' before performing batch inference.
Show answer and explanation
Correct answers: B, C, E
Explanation
To perform batch inference using a scikit-learn model with a Pandas DataFrame, it is critical to ensure the input data matches the training format, including feature names and data structure. Converting the DataFrame to a NumPy array is often required for compatibility, and addressing missing values ensures the process runs smoothly. These steps help avoid common issues during inference.
- A. Incorrect.
This option is incorrect because the model's predict() method cannot be directly applied to a Pandas DataFrame unless the DataFrame is preprocessed to match the format used during training.
- B. Correct.
This option is correct because ensuring that the feature columns in the DataFrame match the feature names used during training is a necessary step for the model to interpret the input correctly.
- C. Correct.
This option is correct because scikit-learn models often expect input data in the form of a NumPy array, so converting the Pandas DataFrame to a NumPy array is advisable.
- D. Incorrect.
This option is incorrect because converting the DataFrame to Pandas is unnecessary if 'new_data' is already a Pandas DataFrame.
- E. Correct.
This option is correct because missing values in the input DataFrame can cause errors or unexpected behavior during batch inference.