MLS-C01 Question 194
Select 4You are training a machine learning model using Amazon SageMaker, and you want to prevent overfitting by applying regularization. You are deciding between using L1 and L2 regularization. Which of the following statements about L1 and L2 regularization are correct?
- A
L1 regularization tends to produce sparse models by driving some feature weights to exactly zero.
- B
L2 regularization penalizes large weights but does not result in sparse models.
- C
L2 regularization is better suited for feature selection because it eliminates irrelevant features.
- D
L1 regularization can be helpful in high-dimensional datasets where many features are irrelevant.
- E
Both L1 and L2 regularization are applied by adding a penalty term to the loss function during training.
Show answer and explanation
Correct answers: A, B, D, E
Explanation
L1 and L2 regularization are techniques used to prevent overfitting by adding penalty terms to the model's loss function. L1 regularization encourages sparsity by driving some feature weights to zero, making it suitable for feature selection. L2 regularization, on the other hand, penalizes large weights without driving them to zero, making it effective for controlling the magnitude of all weights. Understanding when to use each type of regularization is crucial for building robust machine learning models in AWS SageMaker.
- A. Correct.
Correct: L1 regularization encourages sparsity by driving some feature weights to exactly zero, making it effective for feature selection.
- B. Correct.
Correct: L2 regularization penalizes large weights, but unlike L1, it does not result in sparse models. Instead, it reduces the magnitude of all weights proportionally.
- C. Incorrect.
Incorrect: L2 regularization is not typically used for feature selection as it does not drive weights to zero. L1 regularization is more suitable for this purpose.
- D. Correct.
Correct: L1 regularization is effective in high-dimensional datasets with irrelevant features because it can eliminate these features by assigning them a weight of zero.
- E. Correct.
Correct: Both L1 and L2 regularization add a penalty term to the loss function, which discourages overly complex models and helps prevent overfitting.