Databricks Machine Learning Associate Question 600
Select 3You are tasked with evaluating the performance of a binary classification model for predicting whether a customer will churn. The dataset is highly imbalanced, with 95% of customers not churning and only 5% churning. Which metric(s) would be most appropriate to assess the model's performance?
- A
Accuracy
- B
F1 Score
- C
Log Loss
- D
ROC/AUC
- E
Mean Squared Error (MSE)
Show answer and explanation
Correct answers: B, C, D
Explanation
In imbalanced datasets, metrics like F1 Score, Log Loss, and ROC/AUC are better suited than accuracy because they focus on the model's ability to handle the minority class effectively. Accuracy can be misleading in such situations, and MSE is not applicable for classification problems.
- A. Incorrect.
Accuracy is not an appropriate metric for imbalanced datasets since it can be misleading. For example, predicting 'not churn' for all customers would yield 95% accuracy but provide no meaningful insight into the model's performance.
- B. Correct.
F1 Score is suitable for imbalanced datasets because it balances precision and recall, providing a better measure of a model's performance in identifying the positive class (churners) correctly.
- C. Correct.
Log Loss is a good metric for evaluating probabilistic models, as it considers the confidence of the predictions. It is particularly useful in imbalanced datasets where probabilities matter.
- D. Correct.
ROC/AUC is an appropriate metric for imbalanced datasets as it measures the model's ability to distinguish between classes across all classification thresholds, providing a more comprehensive evaluation.
- E. Incorrect.
Mean Squared Error (MSE) is not a classification metric and is typically used for regression problems, making it irrelevant in this scenario.