Databricks Machine Learning Associate Question 249
Select 2You are building a binary classification model to detect fraudulent transactions. The dataset is imbalanced, with only 2% of the transactions being fraudulent. Which evaluation metric(s) would you prioritize to accurately measure the model's ability to identify fraudulent transactions without being biased by the imbalance?
- A
Recall
- B
F1 Score
- C
Accuracy
- D
Precision
- E
Mean Absolute Error (MAE)
Show answer and explanation
Correct answers: A, B
Explanation
In imbalanced classification problems like fraud detection, it is essential to focus on metrics that effectively evaluate the model's ability to identify the minority class (fraudulent transactions). Recall helps ensure that the model identifies as many true positives as possible, while F1 Score provides a balanced measure of precision and recall. Accuracy and MAE are not suitable for this scenario, and precision alone does not address the complete requirements of the problem.
- A. Correct.
Recall is crucial in this scenario because it measures the model's ability to correctly identify all fraudulent transactions (true positives). Since the dataset is imbalanced, recall ensures that the model captures as many positive cases as possible, which is critical for fraud detection.
- B. Correct.
F1 Score balances precision and recall, making it a suitable metric for imbalanced datasets. It gives a single value that considers both false positives and false negatives, making it ideal for evaluating the model in this scenario.
- C. Incorrect.
Accuracy is not a suitable metric for imbalanced datasets because it can be misleading. For example, a model that predicts all transactions as non-fraudulent could achieve high accuracy but fail to identify any fraudulent cases.
- D. Incorrect.
Precision measures how many of the predicted fraudulent transactions are actually fraudulent. While it's important, it does not directly address the need to capture all fraudulent cases, which is critical in this scenario.
- E. Incorrect.
Mean Absolute Error (MAE) is a regression metric and is not applicable for evaluating a binary classification model like this one.