Databricks Machine Learning Associate Question 165
Select 3You are working on a binary classification model in Databricks. The model outputs probabilities for the positive class, and you are tasked with determining whether to use a default threshold of 0.5 or a custom threshold. Which of the following scenarios would require you to set a custom threshold instead of using the default?
- A
The dataset is imbalanced, with a much higher number of negative class samples compared to positive class samples.
- B
The cost of a false positive is significantly higher than the cost of a false negative for your use case.
- C
The model’s performance at the default threshold achieves a high F1 score.
- D
The business objective prioritizes maximizing recall over precision.
- E
The ROC-AUC score of the model is below 0.7, indicating poor overall performance.
Show answer and explanation
Correct answers: A, B, D
Explanation
Adjusting the decision threshold is critical in scenarios where the dataset is imbalanced, the costs of different types of errors are unequal, or the business objective prioritizes specific metrics like recall or precision. The default threshold of 0.5 may not align with these specific requirements, necessitating a custom threshold to optimize the model's performance for the given use case.
- A. Correct.
In imbalanced datasets, the default threshold of 0.5 may not be optimal because the model could favor the majority class, leading to subpar results for the minority class. Adjusting the threshold can help address this issue.
- B. Correct.
When the cost of false positives is significantly higher than false negatives (or vice versa), a custom threshold can help align the predictions with the business priorities.
- C. Incorrect.
If the model achieves a high F1 score at the default threshold, there is no strong need to adjust the threshold as the balance between precision and recall is already optimized.
- D. Correct.
If the business objective prioritizes maximizing recall over precision, a custom threshold lower than 0.5 may be necessary to increase the true positive rate, even at the cost of more false positives.
- E. Incorrect.
The ROC-AUC score reflects the model's ability to distinguish between classes overall but does not indicate whether the threshold should be adjusted. A custom threshold is typically unrelated to ROC-AUC alone.