MLA-C01 Question 201
Select 3You are building a binary classification model to predict whether customers will churn. After training the model, you evaluate its performance using an imbalanced dataset where only 5% of customers have churned. While reviewing the metrics, you notice that the model achieves a high accuracy of 96%. However, the recall for the 'churn' class is very low. Which of the following actions would help you better analyze and improve the model's performance?
- A
Use precision and recall metrics instead of accuracy to evaluate the model.
- B
Generate a confusion matrix to get detailed insights into the classification results.
- C
Use a different dataset with a higher percentage of churn cases to evaluate the model.
- D
Apply SMOTE (Synthetic Minority Oversampling Technique) to balance the dataset before training.
- E
Increase the depth of the decision tree used in the model to improve recall.
Show answer and explanation
Correct answers: A, B, D
Explanation
To analyze model performance on imbalanced datasets, it is crucial to use metrics like precision and recall instead of accuracy, as accuracy can be misleading. Generating a confusion matrix helps in understanding the detailed results for each class. Additionally, rebalancing the dataset using techniques like SMOTE can improve the model's ability to identify minority class instances. These steps collectively help in diagnosing and addressing performance issues.
- A. Correct.
Correct. Precision and recall are better suited for analyzing performance on imbalanced datasets, as they provide insights into how well the model identifies the minority class (e.g., churn). Accuracy is not a reliable metric in this scenario.
- B. Correct.
Correct. A confusion matrix allows you to understand the distribution of true positives, false positives, true negatives, and false negatives, which is critical for analyzing performance, especially for the 'churn' class.
- C. Incorrect.
Incorrect. While using a different dataset with more churn cases might help during training, it does not directly help with analyzing the current model's performance. Instead, rebalancing the data or using better metrics is more appropriate.
- D. Correct.
Correct. SMOTE can be used to balance the dataset by oversampling the minority class (churn cases), which can improve the model's performance and recall for the minority class.
- E. Incorrect.
Incorrect. Simply increasing the depth of a decision tree does not guarantee better recall for the minority class and might lead to overfitting. The issue is primarily related to class imbalance, not the model's complexity.