Databricks Machine Learning Associate Question 397
Select 4A data science team is working on a machine learning project and is evaluating ensemble techniques to improve model performance. They are considering bagging, boosting, and stacking. Which of the following statements correctly describe these techniques?
- A
Bagging reduces variance by training multiple models independently and averaging their predictions.
- B
Boosting reduces bias by sequentially training models, where each subsequent model focuses on the errors of the previous ones.
- C
Stacking combines predictions from multiple base models by training a meta-model on their outputs.
- D
Bagging is more prone to overfitting compared to boosting due to its focus on training sequential learners.
- E
Boosting typically performs better on small, high-bias datasets compared to bagging.
Show answer and explanation
Correct answers: A, B, C, E
Explanation
Bagging, boosting, and stacking are all ensemble learning techniques but serve different purposes. Bagging (Bootstrap Aggregation) reduces variance by training multiple models independently and averaging their predictions. Boosting reduces bias by sequentially training models, where each model corrects the errors of its predecessor. Stacking focuses on leveraging the strengths of multiple models by combining their predictions through a meta-model. Boosting is particularly effective for small, high-bias datasets because it iteratively reduces bias, whereas bagging is designed to address variance and is less prone to overfitting.
- A. Correct.
Correct. Bagging (e.g., Random Forest) reduces variance by training multiple models independently in parallel and aggregating their predictions.
- B. Correct.
Correct. Boosting (e.g., AdaBoost, Gradient Boosting) reduces bias by training models sequentially, with each model improving on the errors of the previous model.
- C. Correct.
Correct. Stacking aggregates the predictions of base models by training a meta-model to make final predictions based on the outputs of the base models.
- D. Incorrect.
Incorrect. Bagging is less prone to overfitting compared to boosting because it trains models independently and uses averaging, which provides regularization.
- E. Correct.
Correct. Boosting often works well on small, high-bias datasets because it focuses on improving errors iteratively, whereas bagging is better suited for reducing variance in larger datasets.