Databricks Machine Learning Associate Question 527
Single answerYou are working on a machine learning pipeline in Databricks and need to preprocess a dataset containing categorical features. One of the columns, 'Region', contains string values such as 'North', 'South', 'East', and 'West'. You decide to use one-hot encoding for this column. Which of the following statements is TRUE about implementing one-hot encoding in Databricks?
- A
One-hot encoding creates a new column for each unique category in the 'Region' column, assigning binary values (0 or 1) to indicate the presence of each category.
- B
One-hot encoding directly replaces the 'Region' column with numerical values (e.g., 1 for 'North', 2 for 'South', etc.) based on the order of unique categories.
- C
One-hot encoding is not supported in Databricks and requires external libraries such as Scikit-learn to process categorical features.
- D
One-hot encoding should only be applied to numerical features, as it does not work with string-based categorical data.
Show answer and explanation
Correct answer: A
Explanation
One-hot encoding is a common preprocessing step for categorical data in machine learning. In Databricks, it creates binary indicator columns for each unique category in a categorical feature. This approach avoids introducing ordinal relationships that could occur with label encoding, making it essential for models that assume numerical features are continuous or ordinal.
- A. Correct.
Correct: One-hot encoding transforms a categorical column into multiple binary indicator columns, one for each unique category, which is the standard behavior of one-hot encoding in Databricks.
- B. Incorrect.
Incorrect: This describes label encoding, not one-hot encoding. Label encoding replaces categories with numerical values, but it can introduce unintended ordinal relationships.
- C. Incorrect.
Incorrect: Databricks supports one-hot encoding through PySpark's
pyspark.ml.feature.OneHotEncoderor similar functionality. External libraries are not required for this task. - D. Incorrect.
Incorrect: One-hot encoding is explicitly designed for categorical features, including string-based categories, and is not limited to numerical features.