MLS-C01 Question 157
Select 3You are training a machine learning model using Amazon SageMaker and need to evaluate its generalization performance. You decide to implement a cross-validation strategy to split your dataset into training and validation sets. Which of the following considerations should you take into account when performing cross-validation?
- A
Ensure that the training and validation sets are mutually exclusive in each fold.
- B
Use the entire dataset as the training set in at least one fold for better performance.
- C
If the dataset is imbalanced, use stratified sampling to maintain class distribution across folds.
- D
Avoid shuffling the dataset before splitting to preserve temporal order in time-series data.
- E
Always use a fixed split ratio such as 80:20, even when performing cross-validation.
Show answer and explanation
Correct answers: A, C, D
Explanation
When performing cross-validation, it is important to ensure that the training and validation sets are mutually exclusive to prevent data leakage. For imbalanced datasets, using stratified sampling ensures representative class distribution in each fold. In time-series data, shuffling should be avoided to preserve temporal dependencies. Fixed split ratios like 80:20 are not a requirement in cross-validation, as the method inherently splits data into dynamic folds.
- A. Correct.
Correct. In cross-validation, the training and validation sets must not overlap to ensure an unbiased evaluation of the model.
- B. Incorrect.
Incorrect. Cross-validation does not use the entire dataset as the training set in any fold. Each fold uses a subset of the data for validation while the rest is used for training.
- C. Correct.
Correct. For imbalanced datasets, stratified sampling ensures that the class distribution in each fold is representative of the overall dataset, improving model evaluation.
- D. Correct.
Correct. For time-series data, shuffling should generally be avoided to preserve the temporal structure, which is critical for meaningful evaluation.
- E. Incorrect.
Incorrect. Cross-validation does not adhere to a fixed split ratio like 80:20. Instead, it dynamically partitions the data into different folds.