Databricks Machine Learning Associate Question 302
Select 4You are tasked with building a machine learning pipeline in Databricks using Spark ML to predict house prices. The dataset contains numeric features (e.g., square footage, number of rooms) and categorical features (e.g., neighborhood). Which of the following steps are necessary to correctly build a pipeline for this task?
- A
Use StringIndexer to convert categorical features into numeric indices
- B
Use VectorAssembler to combine all features into a single feature vector
- C
Normalize numeric features using a StandardScaler before combining them with categorical features
- D
Use Tokenizer to convert numeric features into text format for further processing
- E
Include a regression model like LinearRegression at the end of the pipeline
Show answer and explanation
Correct answers: A, B, C, E
Explanation
Building a Spark ML pipeline involves several key steps: preprocessing categorical features with StringIndexer, normalizing numeric features with StandardScaler, combining features into a single vector using VectorAssembler, and adding a machine learning model at the end of the pipeline. Tokenizer is unnecessary here because the dataset does not include text data.
- A. Correct.
StringIndexer is necessary for converting categorical features into numeric indices that can be used in machine learning models.
- B. Correct.
VectorAssembler is required to combine all the features, both numeric and categorical (after encoding), into a single feature vector for the pipeline.
- C. Correct.
StandardScaler is used to normalize numeric features, which helps improve model performance and convergence, especially for regression tasks.
- D. Incorrect.
Tokenizer is used for processing text data, which is not relevant in this scenario as the dataset does not contain text features.
- E. Correct.
Including a regression model like LinearRegression is essential as it is the final step in the pipeline that performs the prediction task.