Databricks Machine Learning Associate Question 293
Single answerYou are building a machine learning pipeline in Databricks using Spark ML. You include a VectorAssembler in your pipeline to combine several feature columns into a single vector column, followed by a LogisticRegression model to perform binary classification. Which statement best describes the role of the VectorAssembler and LogisticRegression in this pipeline?
- A
VectorAssembler is a Transformer, and LogisticRegression is an Estimator.
- B
VectorAssembler is an Estimator, and LogisticRegression is a Transformer.
- C
Both VectorAssembler and LogisticRegression are Transformers.
- D
Both VectorAssembler and LogisticRegression are Estimators.
Show answer and explanation
Correct answer: A
Explanation
In Spark ML, an Estimator is an algorithm that requires fitting to the data to produce a model (Transformer), while a Transformer directly transforms data without fitting. VectorAssembler is a Transformer that combines multiple columns into a single vector column. LogisticRegression, on the other hand, is an Estimator that fits a model to the data and produces a Transformer for predictions.
- A. Correct.
Correct. VectorAssembler is a Transformer because it transforms input DataFrame columns into a single output vector column. LogisticRegression is an Estimator because it fits a model to the data and produces a Transformer (the fitted model).
- B. Incorrect.
Incorrect. VectorAssembler is not an Estimator; it does not require fitting to data. It directly transforms input columns into a vector column.
- C. Incorrect.
Incorrect. While VectorAssembler is a Transformer, LogisticRegression is not a Transformer. LogisticRegression is an Estimator because it requires fitting to produce a model.
- D. Incorrect.
Incorrect. Both components do not act as Estimators. VectorAssembler is a Transformer, and LogisticRegression acts as an Estimator.