Databricks Machine Learning Associate Question 543
Select 4You are developing a machine learning model in Databricks and notice that your training dataset contains significant class imbalance (e.g., 95% of samples belong to one class). To address this issue, which of the following approaches should you consider during model development?
- A
Use stratified sampling to split the dataset into training and test sets.
- B
Apply SMOTE (Synthetic Minority Oversampling Technique) to generate synthetic samples for the minority class.
- C
Increase the number of training epochs to allow the model to learn minority class patterns more effectively.
- D
Use a weighted loss function to penalize misclassification of the minority class more heavily.
- E
Reduce the size of the majority class by randomly undersampling it.
Show answer and explanation
Correct answers: A, B, D, E
Explanation
Handling class imbalance is critical in model development, as imbalanced datasets can lead to biased models. Approaches such as stratified sampling ensure proper evaluation, while techniques like SMOTE, weighted loss functions, and undersampling directly address the imbalance during model training. Increasing training epochs does not inherently resolve the issue and may lead to overfitting.
- A. Correct.
Stratified sampling ensures that the class distribution in the training and test sets reflects the overall dataset distribution, which is important for evaluating model performance in imbalanced settings.
- B. Correct.
SMOTE is a widely used technique to handle class imbalance by synthetically generating samples for the minority class to balance the dataset.
- C. Incorrect.
Increasing the number of training epochs does not address class imbalance directly. It could lead to overfitting or exacerbate the model's bias towards the majority class.
- D. Correct.
A weighted loss function is a common approach to address class imbalance by assigning higher penalties to errors on the minority class, encouraging the model to focus more on it.
- E. Correct.
Undersampling the majority class can help balance the dataset by reducing the dominance of the majority class, but it may result in loss of valuable data.