Databricks Machine Learning Associate Question 399
Select 3A data scientist is tasked with improving the performance of a classification model. They are considering ensemble learning techniques such as bagging, boosting, and stacking. Which of the following correctly describes the differences between these methods?
- A
Bagging reduces variance by training multiple models independently on bootstrapped samples and averaging their predictions.
- B
Boosting reduces bias by sequentially training models, where each model focuses on correcting the errors of the previous one.
- C
Stacking combines the predictions of multiple base models using a meta-model, which learns how to best combine those predictions.
- D
Bagging is more prone to overfitting than boosting because it emphasizes individual weak learners.
- E
Boosting trains models in parallel, while stacking trains them sequentially.
Show answer and explanation
Correct answers: A, B, C
Explanation
Bagging, boosting, and stacking are popular ensemble learning techniques, but they differ in their approaches and goals. Bagging focuses on reducing variance by training models independently on bootstrapped samples. Boosting reduces bias by training models sequentially, correcting errors iteratively. Stacking combines the predictions of different base models using a meta-model, which learns the best way to aggregate those predictions. Understanding these distinctions is critical for selecting the appropriate ensemble method for a given machine learning problem.
- A. Correct.
Correct: Bagging (Bootstrap Aggregating) reduces variance by training multiple models independently on different subsets of the data (via bootstrapping) and then averaging their predictions for regression or voting for classification.
- B. Correct.
Correct: Boosting reduces bias by training models sequentially, with each model attempting to correct the errors made by the previous models. This iterative approach helps improve overall model performance.
- C. Correct.
Correct: Stacking combines the predictions of multiple base models (trained in parallel) using a meta-model. The meta-model learns to optimally aggregate predictions from the base models.
- D. Incorrect.
Incorrect: Bagging is generally less prone to overfitting compared to boosting because it trains models independently and averages their predictions, making the ensemble more stable.
- E. Incorrect.
Incorrect: Boosting trains models sequentially, not in parallel. Stacking trains base models in parallel and uses a meta-model to combine their outputs.