Databricks Machine Learning Associate Question 609
Single answerYou are training a machine learning model to predict the probability of a customer churning within the next 30 days. The business team requires highly calibrated probability scores to ensure effective decision-making. Which evaluation metric should you prioritize in this scenario?
- A
Accuracy
- B
Precision
- C
Log Loss (Logarithmic Loss)
- D
F1 Score
Show answer and explanation
Correct answer: C
Explanation
In this scenario, the business team requires highly calibrated probability scores to make effective decisions about customer churn. Log Loss is specifically designed to evaluate the quality of probability predictions by penalizing both incorrect and poorly-calibrated predictions. It ensures the predicted probabilities align well with actual outcomes, making it the most appropriate metric in this case.
- A. Incorrect.
Accuracy measures the overall correctness of the model's predictions, but it does not account for the quality of the predicted probabilities, especially when probabilities are required for decision-making.
- B. Incorrect.
Precision focuses on the proportion of true positive predictions out of all positive predictions, but it does not assess the calibration of probability scores, which is the key requirement here.
- C. Correct.
Log Loss evaluates the calibration and confidence of the predicted probabilities. It penalizes predictions that are confident but incorrect, making it the most suitable metric for scenarios requiring well-calibrated probability scores.
- D. Incorrect.
F1 Score is the harmonic mean of precision and recall, focusing on classification performance rather than the quality of probability predictions. It is not appropriate for this scenario where probability calibration is essential.