Databricks Machine Learning Associate Question 171
Select 2You are training a machine learning model in Databricks, and your dataset contains several features with missing values. After imputing these missing values using the mean of the respective columns, why is it important to also create indicator variables to flag the imputed rows?
- A
Indicator variables can help the model differentiate between originally present values and imputed values.
- B
Missing values might carry useful information about the data distribution that the model can learn from.
- C
Adding indicator variables ensures that the imputation process does not introduce bias into the model.
- D
Indicator variables improve the interpretability of the model by directly showing the impact of missing data.
- E
Creating indicator variables prevents the need for imputing missing values altogether.
Show answer and explanation
Correct answers: A, B
Explanation
Adding indicator variables for imputed or replaced missing values allows the model to capture patterns related to missingness. These patterns can be informative and improve model performance, especially if the fact that a value is missing carries predictive significance. Without indicator variables, the imputation process might obscure such patterns by treating all values as equivalent, whether originally present or imputed.
- A. Correct.
Correct. Indicator variables allow the model to distinguish between values that were originally present and those that were imputed, which can improve learning if missingness is informative.
- B. Correct.
Correct. Sometimes missing values themselves can carry meaningful insights about the data's structure or behavior, and indicator variables provide a mechanism for the model to capture this information.
- C. Incorrect.
Incorrect. While indicator variables are useful, they do not inherently prevent bias introduced by the imputation process. The type of imputation method used plays a role in handling bias.
- D. Incorrect.
Incorrect. Indicator variables primarily aid the model's ability to learn patterns related to missingness, but they do not directly improve model interpretability.
- E. Incorrect.
Incorrect. Indicator variables are used alongside imputation methods and do not eliminate the need to handle missing values in the data.