Databricks Machine Learning Associate Question 400
Select 3A data science team is working on improving the predictive performance of their machine learning model. They are considering bagging, boosting, and stacking as ensemble techniques. Which of the following statements correctly describe these techniques?
- A
Bagging reduces variance by training multiple models independently and averaging their predictions.
- B
Boosting reduces bias by sequentially training models, where each model focuses on correcting the errors of the previous one.
- C
Stacking combines predictions from multiple models by training a meta-model on their outputs.
- D
Bagging and boosting both involve a meta-model that learns from the outputs of base models.
- E
Boosting is designed to reduce model variance rather than bias.
Show answer and explanation
Correct answers: A, B, C
Explanation
Bagging, boosting, and stacking are three distinct ensemble learning techniques. Bagging focuses on variance reduction by aggregating independent models trained on resampled datasets. Boosting sequentially trains models to reduce bias by correcting errors iteratively. Stacking combines the outputs of multiple models using a meta-model for better predictive performance. Understanding these distinctions is crucial for selecting the right ensemble approach for a given problem.
- A. Correct.
Correct. Bagging (Bootstrap Aggregating) reduces variance by training multiple models independently on random subsets of the data and averaging their predictions, which helps in stabilizing predictions.
- B. Correct.
Correct. Boosting reduces bias by training models sequentially, with each model focusing on errors made by the prior model, thereby improving overall performance.
- C. Correct.
Correct. Stacking uses a meta-model to learn from the outputs of multiple base models, typically combining their predictions into a final output.
- D. Incorrect.
Incorrect. Bagging does not involve a meta-model; it averages or aggregates predictions directly. Boosting also does not use a meta-model but instead relies on sequential training of models.
- E. Incorrect.
Incorrect. Boosting is primarily designed to reduce bias, not variance, by focusing on improving weak learners iteratively.