AIF-C01 Question 55
Single answerA company has developed a machine learning model for predicting customer churn. The model achieves an accuracy of 92% and an Area Under the ROC Curve (AUC) of 0.85. However, the business team reports that despite the model's strong metrics, the total cost of retaining customers is higher than anticipated. What should the company focus on to address this issue?
- A
Improve the model's accuracy further to reduce churn predictions.
- B
Reevaluate business metrics like cost per retained customer and ROI.
- C
Increase the training dataset to improve model generalization.
- D
Use a different model evaluation metric like F1 score.
Show answer and explanation
Correct answer: B
Explanation
While the model performance metrics like accuracy and AUC are strong, the business problem revolves around the cost of retaining customers. To align the ML model with business objectives, it is crucial to focus on business metrics such as cost per retained customer or ROI instead of solely relying on performance metrics.
- A. Incorrect.
While increasing accuracy might seem beneficial, the issue lies in the business metrics. Focusing solely on accuracy may not address the cost concerns.
- B. Correct.
This is correct. The problem described is related to business metrics rather than model performance metrics. The company should analyze metrics like cost per retained customer and ROI to align the model with business goals.
- C. Incorrect.
Increasing the training dataset may improve the model's performance, but it does not directly address the issue of high retention costs.
- D. Incorrect.
Switching to a different model evaluation metric like F1 score would help in cases of imbalanced datasets, but it won't address the cost concerns directly tied to business metrics.