MLS-C01 Question 160
Single answerYou are building a machine learning model using Amazon SageMaker. You have a dataset with 200,000 records, and you want to evaluate the model's performance using a validation dataset. Which of the following approaches is the MOST appropriate to split the data into training and validation sets while ensuring robust model evaluation?
- A
Randomly split the dataset into 70% for training and 30% for validation once and use these splits for training and evaluation.
- B
Use k-fold cross-validation to create multiple splits of the data, train the model on k-1 folds, and validate it on the remaining fold.
- C
Use the first 70% of the records in the dataset as the training set and the last 30% as the validation set to preserve data order.
- D
Use stratified sampling to ensure that the training and validation datasets contain similar distributions of the target variable.
Show answer and explanation
Correct answer: B
Explanation
K-fold cross-validation is a widely used technique for robust model evaluation in machine learning. It splits the data into k subsets (folds) and ensures that each subset is used for validation exactly once, while the remaining k-1 subsets are used for training. This approach reduces the risk of bias or overfitting due to how the data is partitioned, resulting in more reliable performance metrics for the model.
- A. Incorrect.
Randomly splitting the data once does not ensure robust evaluation because the results may depend heavily on how the split is done, leading to potential bias or variance in the performance metrics.
- B. Correct.
Using k-fold cross-validation is the most robust approach for model evaluation as it ensures that the model is trained and validated on multiple splits of the data, minimizing bias and variance in performance metrics.
- C. Incorrect.
Using the first 70% of records for training and the last 30% for validation risks introducing temporal or positional bias if the data has any inherent order or time dependency.
- D. Incorrect.
While stratified sampling is useful to maintain the target variable's distribution, it does not provide the multiple evaluations and robustness offered by k-fold cross-validation.