Databricks Machine Learning Associate Question 619
Select 3You are training a machine learning model on a dataset with a moderate number of features. During evaluation, you notice that your model has high training accuracy but significantly lower validation accuracy. Which of the following steps would likely help address this issue while considering the bias-variance tradeoff?
- A
Reduce the complexity of the model, such as using fewer layers in a neural network or lowering the maximum depth of a decision tree.
- B
Increase the size of the training dataset to improve generalization.
- C
Add new features to the dataset to provide the model with more information.
- D
Increase the complexity of the model by adding more parameters or layers.
- E
Apply regularization techniques to constrain the model's capacity.
Show answer and explanation
Correct answers: A, B, E
Explanation
The observed high training accuracy and low validation accuracy indicate that the model is overfitting the training data, leading to high variance. To address this, reducing the complexity of the model, increasing the training dataset size, and applying regularization can all help improve generalization and balance the bias-variance tradeoff. Adding irrelevant features or increasing model complexity would not help and could worsen the issue.
- A. Correct.
Reducing the complexity of the model is a valid option as a highly complex model may overfit the training data, leading to a high variance issue. Simplifying the model can help reduce overfitting.
- B. Correct.
Increasing the size of the training dataset helps improve generalization by providing the model with more diverse examples, reducing the variance and improving validation performance.
- C. Incorrect.
Adding new features to the dataset may or may not help. If the features are irrelevant or noisy, they can exacerbate overfitting or increase the complexity unnecessarily. This does not directly address the bias-variance tradeoff in this scenario.
- D. Incorrect.
Increasing the complexity of the model would likely worsen the issue since the model is already overfitting. Higher complexity would increase the variance further, worsening the gap between training and validation accuracy.
- E. Correct.
Applying regularization techniques, such as L1 or L2 regularization, helps constrain the model's capacity and reduces overfitting by penalizing overly complex models. This aligns with addressing the variance issue in the bias-variance tradeoff.