MLS-C01 Question 227
Select 3You are building a binary classification model to predict whether a customer will churn (1) or not churn (0). The dataset is highly imbalanced, with 90% of the customers labeled as non-churners and only 10% labeled as churners. After training the model, you evaluate its performance using different metrics. Which metrics are most appropriate for assessing the performance of this model in this scenario?
- A
Accuracy
- B
Precision
- C
Recall
- D
F1 Score
- E
Root Mean Square Error (RMSE)
Show answer and explanation
Correct answers: B, C, D
Explanation
In scenarios with imbalanced datasets, metrics like accuracy can be misleading because they do not account for the class distribution. Precision and recall are better suited for evaluating the performance of a model in such cases, as they highlight the model's ability to correctly identify positive cases and avoid false positives. The F1 Score combines precision and recall into a single metric, making it particularly useful for imbalanced datasets. RMSE, however, is a regression metric and is irrelevant for binary classification tasks.
- A. Incorrect.
Accuracy may not be an appropriate metric in this scenario because the dataset is highly imbalanced. A model could achieve high accuracy by simply predicting the majority class (non-churners) most of the time, without effectively addressing the minority class (churners).
- B. Correct.
Precision is an important metric when you want to evaluate how many of the predicted positive cases (churners) are actually correct. It is particularly relevant in imbalanced datasets where false positives can have significant consequences.
- C. Correct.
Recall is crucial in this scenario because it measures how well the model identifies actual positive cases (churners). Missing churners (false negatives) could lead to significant business impacts, so recall is highly relevant.
- D. Correct.
F1 Score is the harmonic mean of precision and recall, making it a balanced metric for evaluating performance in imbalanced datasets. It provides a single score that considers both false positives and false negatives.
- E. Incorrect.
Root Mean Square Error (RMSE) is not appropriate for classification tasks like this one. It is a regression metric and does not provide meaningful insights for binary classification problems.