MLS-C01 Question 159
Single answerYou are building a machine learning model on AWS SageMaker using a highly imbalanced dataset. You need to ensure the model generalizes well and does not overfit the training data. Additionally, you want to evaluate the model's performance during training using a validation dataset. Which of the following approaches should you take to properly split the data for training and validation?
- A
Randomly split the dataset into training and validation subsets without considering the class distribution.
- B
Use stratified sampling to split the dataset into training and validation subsets, ensuring that the class distribution is preserved in both.
- C
Use the entire dataset for training and validate the model using the same dataset for evaluation.
- D
Perform K-fold cross-validation to split the dataset into multiple subsets and evaluate the model on different validation folds.
Show answer and explanation
Correct answer: B
Explanation
For an imbalanced dataset, it is essential to ensure that both the training and validation subsets have a similar class distribution. Stratified sampling achieves this by preserving the proportion of classes in both subsets, allowing the model to learn effectively and providing an accurate assessment of its generalization performance. Random splits or improper validation techniques (like training and validating on the same data) can lead to misleading results. K-fold cross-validation is a viable alternative but may not be ideal in all scenarios due to its computational overhead.
- A. Incorrect.
Randomly splitting the data without considering class distribution can lead to an imbalanced validation set, which may provide misleading performance metrics for the model.
- B. Correct.
Using stratified sampling ensures that the class distribution is preserved in both training and validation subsets, which is critical for imbalanced datasets. This helps the model learn from all classes and provides a fair evaluation of its performance.
- C. Incorrect.
Using the same dataset for training and validation will lead to overfitting, as the model will perform well on the training data but may generalize poorly on unseen data.
- D. Incorrect.
While K-fold cross-validation is a robust method for evaluation, it is computationally expensive and may not be necessary if a proper training-validation split is already in place.