Databricks Machine Learning Associate Question 142
Select 2You are working on a machine learning project in Databricks where you aim to predict customer churn. During the Exploratory Data Analysis (EDA) phase, you notice that a particular feature, 'MonthlyCharges,' has several missing values. Which of the following actions would be most appropriate to handle this issue while ensuring proper EDA practices?
- A
Replace missing values in 'MonthlyCharges' with the column's mean or median and document the change.
- B
Drop all rows that contain missing values in 'MonthlyCharges' to avoid introducing bias.
- C
Visualize the distribution of 'MonthlyCharges' to understand its behavior before deciding on handling missing values.
- D
Use a machine learning model to predict and impute the missing values in 'MonthlyCharges' during EDA.
- E
Exclude 'MonthlyCharges' from the dataset entirely to simplify the EDA process.
Show answer and explanation
Correct answers: A, C
Explanation
During EDA, it is critical to handle missing values thoughtfully to ensure that the analysis remains comprehensive and meaningful. Replacing missing values with simple statistics like the mean or median is efficient and allows the feature to be included in further exploration. Additionally, visualizing the distribution of the feature helps in making informed decisions about handling missing values and understanding the data's overall structure.
- A. Correct.
Replacing missing values with the mean or median is a common and valid approach, especially during EDA, as it allows for the inclusion of the feature in further analysis. Documenting the change ensures traceability.
- B. Incorrect.
Dropping rows with missing values might result in loss of important data, especially if the percentage of missing values is small. This should be avoided during EDA unless absolutely necessary.
- C. Correct.
Visualizing the distribution of 'MonthlyCharges' can provide insights into whether the data is skewed or normally distributed, which can inform the choice of imputation strategy. This is a key step in EDA.
- D. Incorrect.
Using a machine learning model to predict missing values is a valid approach, but it is typically done after the EDA phase, as it requires additional modeling and assumptions that go beyond exploratory analysis.
- E. Incorrect.
Excluding the feature entirely during EDA is not recommended as it prevents you from understanding its potential impact on the target variable. This would limit the scope of your analysis.