Databricks Machine Learning Associate Question 556
Select 4You are working on a machine learning pipeline in Databricks and need to preprocess your dataset by scaling features and then train a regression model. Which of the following correctly differentiates between an estimator and a transformer in this context?
- A
An estimator requires a
.fit()method to learn parameters from the data. - B
A transformer directly modifies the input data without learning new parameters.
- C
Both estimators and transformers are used exclusively for training machine learning models.
- D
Transformers generate a transformed dataset by applying a
.transform()method. - E
An estimator can also act as a transformer if it includes a
.transform()method after fitting.
Show answer and explanation
Correct answers: A, B, D, E
Explanation
Estimators and transformers play distinct yet complementary roles in a machine learning pipeline. Estimators are used to learn parameters from data through the .fit() method, while transformers apply these learned parameters to modify or preprocess data using the .transform() method. Some estimators, like scalers, can act as transformers after being fitted, enabling seamless integration into pipelines.
- A. Correct.
Correct: An estimator requires a
.fit()method to learn parameters, such as a regression model learning weights or a scaler learning the mean and standard deviation of the data. - B. Correct.
Correct: A transformer modifies data without learning new parameters. For example, after a scaler has been fitted, it uses those parameters to transform new data.
- C. Incorrect.
Incorrect: Estimators and transformers are not exclusively used for training models. Estimators are used to learn parameters, while transformers are used to preprocess or modify data.
- D. Correct.
Correct: A transformer uses a
.transform()method to apply transformations (e.g., scaling or encoding) to the dataset. - E. Correct.
Correct: Some estimators, such as a fitted scaler, can act as transformers if they include a
.transform()method to modify or preprocess data after fitting.