Databricks Machine Learning Associate Question 523
Select 4You are working on a dataset in Databricks that contains missing values in a column named 'Age'. You decide to impute the missing values. Which of the following approaches are valid for imputing the missing values using the Databricks environment?
- A
Replace missing values in 'Age' with the mean value of the column using PySpark.
- B
Replace missing values in 'Age' with the mode value of the column using the Databricks AutoML feature.
- C
Replace missing values in 'Age' with the median value of the column using the pandas library within a Databricks notebook.
- D
Drop the rows with missing 'Age' values instead of imputing them.
- E
Replace missing values in 'Age' with a constant value of -1 using PySpark.
Show answer and explanation
Correct answers: A, B, C, E
Explanation
Imputation of missing values can be performed using various methods such as the mean, median, mode, or a constant value. Databricks allows for these operations through PySpark, Databricks AutoML, and pandas libraries within notebooks. While dropping rows with missing values is a valid data handling technique, it does not qualify as an imputation approach.
- A. Correct.
This is a valid approach. PySpark provides support for calculating the mean and replacing missing values accordingly in a column.
- B. Correct.
This is a valid approach. Databricks AutoML supports imputing missing values using statistical methods, such as the mode, as part of its preprocessing pipeline.
- C. Correct.
This is a valid approach. Within Databricks notebooks, you can use pandas to calculate the median of a column and replace missing values with it.
- D. Incorrect.
This is not an imputation technique; dropping rows is a method for handling missing values, but it does not involve imputing them.
- E. Correct.
This is a valid approach. PySpark allows replacing missing values with a specific constant value, such as -1, which might be used as a placeholder.