MLA-C01 Question 183
Select 3You are training a neural network model on a dataset with a limited number of samples, and you notice that the model performs well on the training data but poorly on the validation data. Which of the following regularization techniques can help reduce overfitting in this scenario?
- A
Apply dropout to randomly deactivate a subset of neurons during training.
- B
Use weight decay to penalize large weights in the model during optimization.
- C
Increase the size of the neural network to ensure it captures all patterns in the data.
- D
Use L1 regularization to encourage sparsity in the model weights.
- E
Disable batch normalization to reduce computational overhead.
Show answer and explanation
Correct answers: A, B, D
Explanation
Regularization techniques such as dropout, weight decay (L2 regularization), and L1 regularization are effective strategies to reduce overfitting in machine learning models. Dropout prevents reliance on specific neurons, weight decay penalizes large weights, and L1 regularization encourages sparsity. On the other hand, increasing the model size or disabling helpful techniques like batch normalization can worsen overfitting or reduce model performance.
- A. Correct.
Dropout helps reduce overfitting by randomly deactivating neurons during training, forcing the model to learn more robust patterns rather than relying on specific features.
- B. Correct.
Weight decay, a form of L2 regularization, penalizes large weights, which can help prevent overfitting by simplifying the model.
- C. Incorrect.
Increasing the size of the neural network can exacerbate overfitting, especially when training on a limited dataset, as it increases the model's capacity to memorize the training data.
- D. Correct.
L1 regularization drives some weights to zero, encouraging sparsity and helping prevent overfitting by simplifying the model.
- E. Incorrect.
Disabling batch normalization does not help reduce overfitting. In fact, batch normalization can aid in training stability and improve generalization.