MLS-C01 Question 195
Single answerYou are training a machine learning model using Amazon SageMaker. The dataset has a large number of features, some of which are irrelevant or redundant. You notice that the model is overfitting on the training data. Which regularization technique should you use to reduce overfitting while also performing feature selection?
- A
L1 regularization (Lasso)
- B
L2 regularization (Ridge)
- C
Dropout regularization
- D
Batch normalization
Show answer and explanation
Correct answer: A
Explanation
L1 regularization (Lasso) is the correct choice in this scenario because it reduces overfitting while also performing feature selection by driving some feature weights to zero. This is particularly useful when working with datasets that have a large number of features, some of which may be irrelevant or redundant. L2 regularization, dropout, and batch normalization address different aspects of model training but do not provide the feature selection capability that L1 regularization does.
- A. Correct.
L1 regularization (Lasso) adds a penalty equal to the absolute value of the weights to the loss function. This technique encourages sparsity in the model by driving some feature weights to exactly zero, effectively performing feature selection while reducing overfitting.
- B. Incorrect.
L2 regularization (Ridge) adds a penalty equal to the square of the weights to the loss function. While it reduces overfitting by shrinking the weights, it does not encourage sparsity or perform feature selection.
- C. Incorrect.
Dropout regularization is a technique used in neural networks to randomly drop units during training, which reduces overfitting, but it does not perform feature selection.
- D. Incorrect.
Batch normalization is used to normalize the input of each layer in a neural network to speed up training and improve convergence. It does not address overfitting or feature selection.