MLS-C01 Question 158
Single answerYou are building a machine learning model on AWS SageMaker to predict customer churn. Your dataset contains 1 million rows and is imbalanced, with only 10% of the rows labeled as 'churn'. You need to ensure your model is validated effectively and achieves reliable generalization. Which approach should you take to split the data between training and validation?
- A
Use stratified k-fold cross-validation to ensure an even distribution of churn cases in each fold.
- B
Randomly split the dataset into 80% training and 20% validation without considering the imbalance issue.
- C
Use k-fold cross-validation, but without stratification, to test generalization across random folds.
- D
Manually oversample the churn cases in the training set and use a random 80-20 split for validation.
Show answer and explanation
Correct answer: A
Explanation
When working with imbalanced datasets, it is essential to ensure that the minority class is proportionally represented in both training and validation sets. Stratified k-fold cross-validation addresses this by maintaining the class distribution across all folds, leading to more reliable model evaluation and better generalization.
- A. Correct.
Correct: Stratified k-fold cross-validation ensures that each fold has the same proportion of churn cases as the original dataset, which is critical for imbalanced datasets. This approach maintains label distribution consistency across training and validation sets.
- B. Incorrect.
Incorrect: A random split may lead to an uneven distribution of the minority 'churn' class in the training and validation sets, resulting in unreliable model evaluation and poor generalization.
- C. Incorrect.
Incorrect: Non-stratified k-fold cross-validation does not ensure proportional representation of churn cases, potentially leading to folds where the minority class is underrepresented.
- D. Incorrect.
Incorrect: Oversampling the minority class in the training set can lead to overfitting, and a random split for validation does not address the imbalance issue effectively.