MLA-C01 Question 226
Select 3A data science team is using Amazon SageMaker Clarify to identify potential biases in their machine learning model and training dataset. They need to evaluate both pre-training and post-training metrics to detect bias and ensure fairness. Which of the following metrics are relevant when using SageMaker Clarify for bias detection and insights?
- A
Difference in Positive Proportions in Labels (DPPL)
- B
Class Imbalance Ratio
- C
Mean Squared Error
- D
Disparate Impact
- E
Recall
Show answer and explanation
Correct answers: A, B, D
Explanation
Amazon SageMaker Clarify provides both pre-training and post-training bias metrics to help detect and mitigate bias in datasets and models. Pre-training metrics like DPPL and Class Imbalance Ratio analyze the training data, while post-training metrics like Disparate Impact evaluate the fairness of model predictions. SageMaker Clarify does not use general performance metrics like Mean Squared Error or Recall for bias detection.
- A. Correct.
Correct. DPPL is a pre-training bias metric in SageMaker Clarify that measures the difference in positive label proportions across groups to detect bias in the dataset.
- B. Correct.
Correct. Class Imbalance Ratio is a pre-training bias metric in SageMaker Clarify that checks for imbalances in the class distribution of the dataset.
- C. Incorrect.
Incorrect. Mean Squared Error is a performance metric used to evaluate regression models, not a bias detection metric provided by SageMaker Clarify.
- D. Correct.
Correct. Disparate Impact is a post-training bias metric in SageMaker Clarify that measures the ratio of favorable outcomes between groups to assess fairness in model predictions.
- E. Incorrect.
Incorrect. Recall is a performance metric used during model evaluation to measure how well the model identifies positive instances, but it is unrelated to bias detection in SageMaker Clarify.