MLS-C01 Question 193
Select 2You are training a linear regression model in Amazon SageMaker. During experimentation, you notice that the model is overfitting the training data. To address this, you decide to apply regularization. Which of the following regularization techniques can help reduce overfitting in this scenario?
- A
L1 regularization, which adds the absolute value of the weights to the loss function
- B
L2 regularization, which adds the squared value of the weights to the loss function
- C
Dropout, which randomly sets a fraction of the model’s parameters to zero during training
- D
Batch normalization, which normalizes the input features to speed up convergence
- E
Using a higher learning rate to penalize large weight updates
Show answer and explanation
Correct answers: A, B
Explanation
L1 and L2 regularization are standard techniques to mitigate overfitting in linear regression models. L1 regularization encourages sparsity by driving some weights to zero, while L2 regularization penalizes large weights to reduce model complexity. Both techniques modify the loss function to include a penalty term, helping to generalize the model better to unseen data.
- A. Correct.
L1 regularization, also known as Lasso regression, helps reduce overfitting by penalizing large weights and can also lead to sparse models by driving some weights to zero.
- B. Correct.
L2 regularization, also known as Ridge regression, helps reduce overfitting by penalizing large weights but does not drive weights to zero. It is a common technique for controlling overfitting.
- C. Incorrect.
Dropout is a regularization technique used specifically in neural networks, not linear regression. It is not applicable in this scenario.
- D. Incorrect.
Batch normalization is used for stabilizing and accelerating training in deep learning models, not for regularization in linear regression models.
- E. Incorrect.
Using a higher learning rate may destabilize training and lead to poor convergence. It is not a method for addressing overfitting.