MLA-C01 Question 116
Select 3You are building a machine learning model to predict customer churn for a subscription-based service. During model evaluation on the validation dataset, you notice that the model achieves high accuracy but performs poorly on identifying customers who are likely to churn. Which of the following actions should you take to improve the model's performance on identifying churned customers?
- A
Change the evaluation metric to F1-score or precision-recall to better assess performance on the churned class.
- B
Add more features to the dataset to improve the model's ability to differentiate between churned and non-churned customers.
- C
Use oversampling techniques like SMOTE to handle the class imbalance in the dataset.
- D
Lower the decision threshold for the churn prediction to increase sensitivity for the churned class.
- E
Switch to using a regression model instead of a classification model for predicting churn.
Show answer and explanation
Correct answers: A, C, D
Explanation
In this scenario, the issue stems from the class imbalance and the need to better identify the minority class (churned customers). Changing the evaluation metric (F1-score or precision-recall) ensures the model is evaluated appropriately for this imbalanced dataset. Oversampling techniques like SMOTE help the model learn patterns in the minority class, and lowering the decision threshold increases sensitivity toward the churned class. Adding more features or switching to a regression model does not directly address the problem.
- A. Correct.
Changing the evaluation metric to F1-score or precision-recall is appropriate when dealing with imbalanced datasets, as these metrics focus on the trade-off between precision and recall, which is critical for the minority class (churned customers).
- B. Incorrect.
Adding more features can improve model performance in general, but it does not directly address the issue of poor performance on the minority class, which is the focus of the problem.
- C. Correct.
Using oversampling techniques like SMOTE is a standard approach to handle class imbalance, helping the model pay more attention to the underrepresented class (churned customers).
- D. Correct.
Lowering the decision threshold increases the model's sensitivity (recall) for the minority class, which makes the model better at identifying churned customers.
- E. Incorrect.
Switching to a regression model is not appropriate because churn prediction is inherently a classification problem (whether the customer will churn or not).