Databricks Machine Learning Associate Question 143
Select 3You are working on a dataset in Databricks and performing exploratory data analysis (EDA). The dataset contains missing values, highly skewed features, and categorical variables. Which of the following steps are appropriate to include in your EDA to prepare the dataset for machine learning?
- A
Visualize the distribution of numerical features to identify skewness and outliers
- B
Replace missing values by directly dropping rows with null values
- C
Analyze pairwise correlations between numerical features to detect multicollinearity
- D
Perform one-hot encoding on categorical variables to convert them into numerical form
- E
Train a machine learning model before addressing missing values and skewness
Show answer and explanation
Correct answers: A, C, D
Explanation
EDA involves analyzing and preprocessing the dataset to detect and address issues such as missing values, skewness, multicollinearity, and categorical variables. Visualizing distributions, checking correlations, and encoding categorical variables are essential steps. However, dropping rows with null values may lead to data loss, and training a model before proper preprocessing can result in poor outcomes.
- A. Correct.
Visualizing the distribution of numerical features helps identify skewness, outliers, and other patterns in the data, which is a key part of EDA.
- B. Incorrect.
Dropping rows with null values is not always the best approach, as it may lead to loss of valuable data. Alternative imputation techniques should also be considered.
- C. Correct.
Analyzing pairwise correlations between numerical features can help identify multicollinearity, which is important for feature selection and reducing redundancy.
- D. Correct.
One-hot encoding is necessary to convert categorical variables into a format that can be understood by machine learning algorithms.
- E. Incorrect.
Training a model before addressing data issues such as missing values and skewness can lead to biased or inaccurate results and is not recommended during EDA.