Databricks Machine Learning Associate Question 185
Select 3You are working on a machine learning project in Databricks where the dataset contains several numerical features with missing values. You decide to impute the missing values using the mean or median of each feature. Which of the following statements are true about this approach?
- A
Imputing missing values with the mean or median can help preserve the overall distribution of the data.
- B
Mean imputation is sensitive to outliers, while median imputation is more robust in the presence of outliers.
- C
Using mean or median imputation guarantees that the model's performance will improve.
- D
This approach is only applicable to numerical features and not categorical features.
- E
Imputing missing values with the mean or median is a preprocessing step that should be applied before splitting the dataset into training and test sets.
Show answer and explanation
Correct answers: A, B, D
Explanation
Imputing missing values with the mean or median is a common preprocessing step for handling numerical features with missing data. It helps maintain the dataset's structure but has limitations, such as sensitivity to outliers (in the case of mean imputation) and the inability to guarantee improved model performance. Additionally, this technique is specific to numerical data and must be applied correctly in the machine learning workflow to avoid data leakage.
- A. Correct.
Correct: Imputing missing values with the mean or median can help maintain the overall data distribution, especially when the missingness is random.
- B. Correct.
Correct: Mean imputation is affected by extreme values (outliers), whereas the median is more robust and less influenced by outliers.
- C. Incorrect.
Incorrect: While imputing missing values can improve the quality of the data, there is no guarantee that it will lead to better model performance in all cases.
- D. Correct.
Correct: Mean and median imputation are only applicable to numerical features since they involve mathematical calculations that do not apply to categorical data.
- E. Incorrect.
Incorrect: Missing value imputation should be applied after splitting the dataset into training and test sets to avoid data leakage.