Databricks Generative AI Engineer Associate Question 182
Select 3You are tasked with building a chain in Databricks that uses a PyFunc model for performing text sentiment analysis. The chain includes pre-processing to clean and tokenize the text input and post-processing to convert numeric model outputs into sentiment labels (e.g., 'positive', 'negative', 'neutral'). Which of the following steps are necessary to implement this pipeline correctly?
- A
Define a custom PyFunc class that implements both pre-processing and post-processing logic.
- B
Save the pre-processing and post-processing scripts as separate Delta tables for reusability.
- C
Deploy the PyFunc model using MLflow after adding the pre-processing and post-processing steps.
- D
Ensure the pre-processing and post-processing steps are included in the PyFunc model's predict method.
- E
Use a Databricks SQL query to apply pre- and post-processing outside the PyFunc model.
Show answer and explanation
Correct answers: A, C, D
Explanation
To implement a PyFunc model chain with pre- and post-processing, the entire pipeline (pre-processing, model inference, and post-processing) must be encapsulated in the PyFunc class. This ensures the model can be deployed and served as a single unit using MLflow. Defining pre- and post-processing steps outside the PyFunc model or relying on unrelated tools (e.g., Delta tables, SQL queries) is not an appropriate approach for this use case.
- A. Correct.
Correct: A custom PyFunc class must be defined to encapsulate both pre-processing and post-processing logic along with the model's prediction code.
- B. Incorrect.
Incorrect: While Delta tables are useful for storing data transformations, they are not relevant to implementing pre-processing and post-processing in a PyFunc pipeline.
- C. Correct.
Correct: Deploying the PyFunc model with MLflow ensures the entire pipeline (including pre- and post-processing) is versioned and can be served for inference.
- D. Correct.
Correct: The PyFunc class should override the predict method, where pre-processing, model inference, and post-processing are all implemented.
- E. Incorrect.
Incorrect: Pre- and post-processing should be implemented within the PyFunc model rather than relying on external SQL queries, which would break the chain's encapsulation.