Databricks Machine Learning Associate Question 597
Select 3You are tasked with evaluating a machine learning classification model for predicting whether a customer will churn or not. The dataset is imbalanced, with only 10% of customers labeled as 'churn'. Which of the following evaluation metrics would be the most appropriate to assess the model's performance?
- A
Accuracy
- B
F1 Score
- C
Log Loss
- D
ROC AUC
Show answer and explanation
Correct answers: B, C, D
Explanation
In imbalanced classification tasks, metrics like F1 Score, Log Loss, and ROC AUC are better suited than accuracy because they provide a more nuanced evaluation of the model's performance. F1 Score balances precision and recall, Log Loss evaluates probabilistic predictions, and ROC AUC measures the ability to discriminate between classes. Accuracy, on the other hand, can be misleading in such scenarios as it does not account for class imbalance.
- A. Incorrect.
Accuracy is not an appropriate metric for imbalanced datasets, as it can be misleading. For instance, predicting all customers as 'non-churn' would yield a high accuracy (90%) but would completely fail to identify any churn cases.
- B. Correct.
F1 Score is a good metric for imbalanced datasets as it considers both precision and recall, providing a balanced measure of a model's ability to correctly classify the minority class ('churn' in this case).
- C. Correct.
Log Loss is suitable because it evaluates the probabilistic predictions of the model, penalizing incorrect confident predictions more than less confident ones, which is critical for imbalanced datasets.
- D. Correct.
ROC AUC is effective for imbalanced datasets as it assesses the model's ability to distinguish between the positive ('churn') and negative ('non-churn') classes, regardless of class imbalance.