MLS-C01 Question 199
Select 3You are building a machine learning model to predict customer churn using an imbalanced dataset. To evaluate the model's performance, you decide to use k-fold cross-validation. Which of the following steps are necessary to ensure the cross-validation process is both reliable and effective in this scenario?
- A
Use stratified k-fold cross-validation to maintain the class distribution across folds.
- B
Randomly shuffle the data before splitting it into folds to minimize bias.
- C
Ensure that each fold includes the same number of samples for each feature.
- D
Use a separate validation set outside of the cross-validation process for hyperparameter tuning.
- E
Increase the value of k in k-fold cross-validation to reduce variance in model evaluation results.
Show answer and explanation
Correct answers: A, B, D
Explanation
In this scenario, stratified k-fold cross-validation is essential to handle the imbalance in the dataset by preserving the class distribution across folds. Randomly shuffling the data ensures unbiased splits, while using a separate validation set for hyperparameter tuning avoids data leakage. Increasing the value of k is optional and depends on the specific requirements of the model and dataset.
- A. Correct.
Stratified k-fold cross-validation ensures that the class distribution of the target variable is preserved in each fold, which is crucial for imbalanced datasets.
- B. Correct.
Randomly shuffling the data before splitting into folds helps reduce bias and ensures that the folds are representative of the entire dataset.
- C. Incorrect.
Ensuring that each fold includes the same number of samples for each feature is not necessary in k-fold cross-validation; instead, preserving the class distribution is more important.
- D. Correct.
Using a separate validation set outside of the cross-validation process is necessary for hyperparameter tuning to avoid data leakage and ensure an unbiased evaluation of the model.
- E. Incorrect.
Increasing the value of k in k-fold cross-validation can reduce variance but is not a mandatory step for effective cross-validation. The choice of k depends on the dataset size and computational constraints.