Databricks Machine Learning Associate Question 557
Select 2You are working on a machine learning pipeline in Databricks and need to preprocess your dataset before training a model. You plan to use a StringIndexer to convert categorical columns into numerical indices and a VectorAssembler to combine multiple features into a single vector column. Which of the following statements correctly describes the roles of StringIndexer and VectorAssembler in the pipeline?
- A
StringIndexer is an estimator that fits on the dataset to determine the mapping of categories to indices.
- B
StringIndexer is a transformer that directly converts categorical columns into numerical indices without needing to fit on the dataset.
- C
VectorAssembler is a transformer that combines multiple feature columns into a single vector column.
- D
VectorAssembler is an estimator that requires fitting on the dataset to generate combined vector columns.
- E
Both StringIndexer and VectorAssembler are transformers and do not require fitting on the dataset.
Show answer and explanation
Correct answers: A, C
Explanation
In the context of Databricks machine learning pipelines, an estimator is a stage that requires fitting on the dataset to produce a transformer, while a transformer directly applies a transformation to the dataset. StringIndexer is an estimator because it needs to compute the mapping of categorical values to indices before transforming data. On the other hand, VectorAssembler is a transformer because it directly combines feature columns into a vector without the need for fitting.
- A. Correct.
StringIndexer is an estimator because it needs to fit on the dataset to compute the mapping between categorical values and numerical indices. After fitting, it produces a transformer to perform the actual transformation.
- B. Incorrect.
This is incorrect because StringIndexer is not a transformer by itself; it must first fit on the data to create a transformer.
- C. Correct.
VectorAssembler is a transformer because it does not require fitting. It directly transforms input feature columns into a single vector column.
- D. Incorrect.
This is incorrect because VectorAssembler does not require fitting on the dataset; it directly performs the transformation.
- E. Incorrect.
This is incorrect because only VectorAssembler is a transformer. StringIndexer is an estimator and requires fitting on the dataset.