MLA-C01 Question 213
Select 3A data science team is building a binary classification model to detect fraudulent transactions. During the evaluation phase, the team observes that the dataset is highly imbalanced, with only 1% of the transactions being fraudulent. Which evaluation metric(s) should the team prioritize to assess the model's performance effectively?
- A
Accuracy
- B
Precision
- C
Recall
- D
F1 Score
- E
Root Mean Square Error (RMSE)
Show answer and explanation
Correct answers: B, C, D
Explanation
For imbalanced datasets like the one in this scenario, metrics such as precision, recall, and F1 Score are more indicative of a model's performance than accuracy. Accuracy can be misleading since predicting the majority class (non-fraudulent) most of the time can result in high accuracy but poor fraud detection. Precision and recall focus on the minority class, while the F1 Score balances these metrics to provide a single evaluation measure.
- A. Incorrect.
Accuracy is not a reliable metric for imbalanced datasets because a model can achieve high accuracy by simply predicting the majority class (non-fraudulent) most of the time, without effectively identifying the minority class (fraudulent).
- B. Correct.
Precision is important for identifying fraudulent transactions in this case because it measures the proportion of correctly identified fraudulent transactions (true positives) out of all predicted fraudulent transactions. It helps minimize false positives.
- C. Correct.
Recall is critical in this scenario as it measures the proportion of actual fraudulent transactions (true positives) that are correctly identified by the model. Since missing fraudulent transactions could have significant consequences, high recall is desirable.
- D. Correct.
F1 Score is the harmonic mean of precision and recall. It provides a balanced measure that is particularly useful for imbalanced datasets, as it considers both false positives and false negatives.
- E. Incorrect.
Root Mean Square Error (RMSE) is not suitable for evaluating classification models. It is a regression metric that measures the average magnitude of errors in continuous predictions.