Databricks Machine Learning Associate Question 172
Select 3You are building a machine learning model using a dataset that contains missing values in several columns. After imputing the missing values with the mean, you decide to add indicator variables (binary flags) to specify whether a value was imputed. Why is this an important step?
- A
It allows the model to capture potential patterns associated with missingness that might be predictive.
- B
It ensures the model can differentiate between original and imputed values, which might have different distributions.
- C
It prevents the imputed values from introducing bias in the model's predictions.
- D
It helps reduce the dimensionality of the dataset by removing the need for imputation.
- E
It ensures compliance with data privacy standards by masking missing values.
Show answer and explanation
Correct answers: A, B, C
Explanation
Adding indicator variables is important because missingness itself can carry predictive information, and differentiating between original and imputed values can improve model performance and reduce bias. This step ensures the model can learn any patterns associated with missing data without treating imputed values as equivalent to true observations.
- A. Correct.
Adding indicator variables allows the model to capture any potential predictive power of the fact that a value was missing, as missingness itself could be informative.
- B. Correct.
Indicator variables help the model distinguish between original and imputed values, which could have different statistical properties and impact the model's learning.
- C. Correct.
Indicator variables can help mitigate bias by explicitly accounting for missing data, ensuring that the model doesn't treat imputed values as if they were true observations.
- D. Incorrect.
Adding indicator variables does not reduce dimensionality; instead, it increases it by adding additional features.
- E. Incorrect.
Indicator variables are not directly related to data privacy; they serve to provide more information about the dataset's structure and missingness.