MLA-C01 Question 91
Select 2You are building a machine learning model to classify customer feedback as 'positive', 'negative', or 'neutral'. Upon examining the training dataset, you observe that 70% of the labels are 'positive', 20% are 'neutral', and 10% are 'negative'. Which pre-training bias metric(s) would be most relevant to evaluate this issue and its potential impact on model performance?
- A
Class Imbalance (CI)
- B
Difference in Proportions of Labels (DPL)
- C
Mean Absolute Error (MAE)
- D
Confusion Matrix Analysis
- E
Data Drift Analysis
Show answer and explanation
Correct answers: A, B
Explanation
Class imbalance (CI) and Difference in Proportions of Labels (DPL) are critical pre-training bias metrics when working with categorical datasets where label distribution is uneven. They help identify imbalances that could lead to biased model predictions, such as over-predicting the majority class ('positive' in this scenario) and underperforming on minority classes ('negative' and 'neutral'). Addressing these metrics can improve model fairness and accuracy across all classes.
- A. Correct.
Class Imbalance (CI) is directly relevant as it measures the disproportionality of label frequencies in the dataset. In this case, the high prevalence of 'positive' labels indicates a significant class imbalance.
- B. Correct.
Difference in Proportions of Labels (DPL) is relevant because it quantifies the differences in label distributions, which helps identify whether any specific label is disproportionately represented compared to others.
- C. Incorrect.
Mean Absolute Error (MAE) is a performance metric for regression models, not a bias metric for analyzing class distributions in categorical datasets.
- D. Incorrect.
Confusion Matrix Analysis is a post-training evaluation technique that measures model performance but does not help identify pre-training biases like class imbalance or label distribution issues.
- E. Incorrect.
Data Drift Analysis is used to detect changes in data distribution between training and inference data, but it is not used for identifying biases in pre-training datasets.