MLA-C01 Question 200
Single answerYou are training a binary classification model using Amazon SageMaker, and the model achieves an accuracy of 92% on the test dataset. However, during a review, the business stakeholders highlight that the model is misclassifying a critical minority class that makes up only 5% of the dataset. Which metric should you prioritize to better evaluate the model's performance in this scenario?
- A
Precision
- B
Recall
- C
F1 Score
- D
Accuracy
Show answer and explanation
Correct answer: B
Explanation
In scenarios with imbalanced datasets, especially where a minority class is critical, accuracy is not a reliable metric as it can obscure poor performance on the minority class. Recall is the most important metric here because it directly evaluates how many of the actual minority class instances are correctly identified. Precision and F1 Score are useful but secondary in this case, as the primary need is to ensure the minority class is not overlooked.
- A. Incorrect.
Precision measures the proportion of true positive predictions out of all positive predictions. While it is important, it does not specifically address the problem of identifying the minority class.
- B. Correct.
Recall measures the proportion of true positive predictions out of all actual positives. It is critical in this scenario because the goal is to ensure that the minority class instances are correctly identified, even at the expense of some false positives.
- C. Incorrect.
F1 Score is the harmonic mean of precision and recall, but it balances the two. In this case, focusing solely on recall is more appropriate to address the issue with the minority class.
- D. Incorrect.
Accuracy measures the overall correctness of the model but is not suitable in imbalanced datasets because it can be misleading. A model that predicts the majority class well can still achieve high accuracy even if it fails to identify the minority class.