MLA-C01 Question 181
Select 3You are training a deep learning model on a complex dataset, but you notice that the model is overfitting to the training data. Which of the following regularization techniques can help reduce overfitting and improve the model's generalization ability?
- A
Apply dropout to randomly deactivate a fraction of neurons during training.
- B
Increase the size of the training dataset by generating synthetic data.
- C
Use weight decay to penalize large weights in the loss function.
- D
Implement L1 regularization to encourage sparsity in the model's weights.
- E
Decrease the number of layers in the model architecture.
Show answer and explanation
Correct answers: A, C, D
Explanation
Regularization techniques like dropout, weight decay (L2 regularization), and L1 regularization are effective in reducing overfitting by imposing constraints on the model's complexity or altering the training process. These techniques improve generalization by preventing the model from learning overly complex patterns that do not generalize well to unseen data. While other strategies like increasing dataset size or simplifying the model architecture can also reduce overfitting, they are not specifically classified as regularization techniques.
- A. Correct.
Dropout is a regularization technique that deactivates a fraction of neurons during training to reduce overfitting by preventing co-adaptation of neurons.
- B. Incorrect.
Increasing the size of the dataset is a data augmentation strategy, not a regularization technique. While it helps reduce overfitting, it is not directly related to regularization.
- C. Correct.
Weight decay, also known as L2 regularization, penalizes large weights, thus helping to reduce overfitting by discouraging complex models.
- D. Correct.
L1 regularization encourages sparsity by driving some weight parameters to zero, which can help improve generalization.
- E. Incorrect.
Decreasing the number of layers is a model simplification strategy, not a regularization technique. While it might reduce overfitting, it does so by reducing model capacity rather than by applying regularization.