MLS-C01 Question 243
Select 3You are building a machine learning model on Amazon SageMaker and need to ensure that the model generalizes well to unseen data. To achieve this, you aim to perform cross-validation. Which of the following approaches would correctly implement cross-validation in SageMaker?
- A
Use k-fold cross-validation by splitting the dataset into k subsets, training the model on k-1 subsets, and testing it on the remaining subset in a loop.
- B
Use SageMaker’s built-in cross-validation algorithm to automatically split the dataset and train multiple models.
- C
Manually split your dataset into training and validation sets multiple times, train the model on each training set, and evaluate it on the corresponding validation set.
- D
Use Amazon SageMaker Processing to script the k-fold splitting of the dataset, and then train the model using SageMaker Training Jobs in each fold.
- E
Enable the 'cross_validation' parameter in the SageMaker built-in algorithms to automatically perform cross-validation.
Show answer and explanation
Correct answers: A, C, D
Explanation
Cross-validation is a technique used to evaluate the generalizability of a machine learning model by splitting the dataset into multiple subsets for training and validation. In SageMaker, cross-validation is not handled automatically by built-in algorithms, so you need to implement it manually or use tools like SageMaker Processing to script the process. The correct answers include approaches that align with these best practices.
- A. Correct.
Correct: This is the standard approach to k-fold cross-validation, where the dataset is split into k subsets, and the model is trained and validated k times, each time on a different combination of training and validation subsets. This can be implemented manually or scripted in SageMaker.
- B. Incorrect.
Incorrect: SageMaker does not have a built-in cross-validation algorithm. Cross-validation must be implemented manually or using custom scripts.
- C. Correct.
Correct: Manually splitting the dataset into training and validation sets for multiple iterations is a valid way to perform cross-validation. This approach can be implemented using SageMaker notebooks.
- D. Correct.
Correct: SageMaker Processing can be used to script the k-fold splitting of the dataset. Afterward, SageMaker Training Jobs can be used to train the model for each fold, which is a valid cross-validation implementation.
- E. Incorrect.
Incorrect: SageMaker’s built-in algorithms do not have a 'cross_validation' parameter. Cross-validation needs to be implemented manually or using custom scripts.