MLS-C01 Question 198
Select 3You are building a machine learning model using Amazon SageMaker. To evaluate the model's performance, you decide to implement k-fold cross-validation. Which of the following steps must you perform to ensure proper cross-validation and avoid data leakage?
- A
Split the dataset into k subsets, ensuring that each subset has a similar distribution of the target variable.
- B
Ensure that the test dataset used for the final evaluation is not included in any of the k folds.
- C
Use the entire training dataset as a single fold in each iteration to maximize training data usage.
- D
Ensure that data preprocessing steps, such as normalization, are applied independently to each fold during training and validation.
- E
Use the same random seed for all splits to ensure reproducibility.
Show answer and explanation
Correct answers: A, B, D
Explanation
To perform k-fold cross-validation effectively, it is crucial to split the dataset into k subsets with a consistent target distribution, avoid data leakage by keeping the test dataset separate, and apply preprocessing independently to each fold. These practices help ensure unbiased and reliable performance evaluation of the machine learning model. Using the entire dataset as a single fold or relying on random seeds does not align with proper cross-validation methodology.
- A. Correct.
Correct: When performing k-fold cross-validation, it is important to split the dataset in a way that maintains the distribution of the target variable (e.g., stratified sampling) to ensure consistent and unbiased evaluation.
- B. Correct.
Correct: To properly evaluate model performance, the test dataset for final evaluation should be kept separate from the folds used during cross-validation to prevent data leakage.
- C. Incorrect.
Incorrect: Using the entire training dataset as a single fold in each iteration is not valid cross-validation. k-fold cross-validation requires dividing the dataset into k subsets, with k-1 subsets used for training and 1 subset used for validation in each iteration.
- D. Correct.
Correct: Data preprocessing steps, such as normalization or scaling, should be applied independently to each fold to prevent information from the validation set leaking into the training set.
- E. Incorrect.
Incorrect: While using the same random seed can ensure reproducibility, it is not a required step for proper cross-validation. It is a best practice for consistent results but does not directly impact the correctness of cross-validation.