MLS-C01 Question 65
Select 3A data scientist is analyzing a dataset containing millions of records to predict customer churn for a subscription-based service. They notice that the dataset contains missing values in several key features, as well as anomalies like extreme outliers. Additionally, some categorical features have a high cardinality. The data scientist wants to ensure the dataset is properly prepared for machine learning. Which actions should the data scientist take during the exploratory data analysis (EDA) phase to address these issues?
- A
Impute missing values using statistical methods or domain knowledge.
- B
Remove outliers without any analysis to avoid data contamination.
- C
Perform feature engineering to reduce the cardinality of categorical features.
- D
Scale numerical features to normalize their range before addressing missing values.
- E
Generate visualizations like box plots and histograms to understand the distribution of the data.
Show answer and explanation
Correct answers: A, C, E
Explanation
During exploratory data analysis, the goal is to understand the dataset and prepare it for machine learning. Handling missing values appropriately, addressing high cardinality in categorical features, and using visualizations to assess the data distribution are all critical steps. Removing outliers without analysis and scaling features prematurely are not best practices for EDA.
- A. Correct.
Correct: Imputing missing values is a key step in handling incomplete data during EDA. Using statistical methods or domain knowledge ensures the missing values are handled in a meaningful way.
- B. Incorrect.
Incorrect: Outliers should not be removed blindly. Instead, their impact should be analyzed first, as they might contain important information or indicate data quality issues.
- C. Correct.
Correct: High cardinality in categorical features can lead to inefficiencies and overfitting in machine learning models. Feature engineering techniques, such as grouping or encoding, can help address this issue.
- D. Incorrect.
Incorrect: Scaling numerical features is typically done after handling missing values, not before. Scaling is a preprocessing step for certain machine learning algorithms but is not directly part of EDA.
- E. Correct.
Correct: Visualizations like box plots and histograms provide insights into data distributions, outliers, and potential issues, which are crucial during the EDA phase.