MLA-C01 Question 209
Select 2A data scientist is building a binary classification model to identify fraudulent transactions. After training the model, they evaluate its performance using the confusion matrix and calculate additional metrics. The model has the following confusion matrix:
| Predicted Fraud | Predicted Not Fraud | |
|---|---|---|
| Actual Fraud | 80 | 20 |
| Actual Not Fraud | 10 | 190 |
Which evaluation metric(s) should the data scientist prioritize to assess the model's ability to correctly identify fraudulent transactions while minimizing false negatives?
- A
Recall
- B
Precision
- C
Accuracy
- D
F1 Score
- E
Root Mean Square Error (RMSE)
- F
Area Under the ROC Curve (AUC)
Show answer and explanation
Correct answers: A, D
Explanation
For fraud detection, minimizing false negatives (missed fraudulent transactions) is often more critical than minimizing false positives. Recall is the most relevant metric in this case, as it captures the ability to identify actual fraudulent transactions. Additionally, the F1 Score is useful for balancing precision and recall, especially in imbalanced datasets where both metrics are important. Metrics like accuracy or RMSE are less relevant for this specific scenario.
- A. Correct.
Recall measures the proportion of actual positive cases (fraudulent transactions) correctly identified by the model. It is crucial when false negatives (missed fraudulent transactions) must be minimized.
- B. Incorrect.
Precision measures the proportion of predicted positive cases (fraudulent transactions) that are actually positive. While important, it is not the primary metric when minimizing false negatives.
- C. Incorrect.
Accuracy measures the overall correctness of the model's predictions. However, in imbalanced datasets (e.g., fraud detection), accuracy can be misleading and may not reflect the model's ability to identify minority class instances.
- D. Correct.
F1 Score is the harmonic mean of precision and recall. It is particularly useful when there is an imbalance between classes and when both false positives and false negatives are critical. It combines precision and recall into a single metric.
- E. Incorrect.
Root Mean Square Error (RMSE) is used for regression problems and does not apply to binary classification tasks.
- F. Incorrect.
Area Under the ROC Curve (AUC) measures the trade-off between true positive rate and false positive rate across different thresholds. While it provides a broader view of model performance, it does not directly focus on minimizing false negatives.