Databricks Machine Learning Associate Question 59
Single answerYou are working on a classification problem involving customer churn prediction. You decide to use Databricks AutoML to expedite the model development process. After running AutoML on your dataset, you receive multiple model candidates, including their evaluation metrics and feature importance scores. What is the next best step to take to ensure you select the most appropriate model?
- A
Select the model with the highest accuracy, as it indicates the best performance across all classes.
- B
Evaluate the models further by considering metrics like precision, recall, or F1-score, depending on the business objective.
- C
Choose the model with the simplest architecture to minimize computational costs during inference.
- D
Directly deploy the top-performing model recommended by AutoML without further evaluation.
Show answer and explanation
Correct answer: B
Explanation
When using Databricks AutoML, it is important to evaluate the shortlisted models based on metrics that align with the business objectives. For a classification problem like customer churn prediction, precision, recall, or F1-score might be more relevant than just accuracy, especially if the dataset is imbalanced. This ensures the model selected aligns with the real-world impact of false positives or false negatives in the use case.
- A. Incorrect.
Accuracy alone may not be the best metric for classification problems, especially if the dataset is imbalanced. Additional metrics like precision, recall, or F1-score should be considered based on the specific use case.
- B. Correct.
This is the correct answer. Different business objectives may require focusing on specific evaluation metrics. For example, in a churn prediction scenario, recall might be prioritized to identify as many churners as possible.
- C. Incorrect.
While simpler models can reduce computational costs, model performance in terms of metrics relevant to the business problem should take precedence before considering simplicity.
- D. Incorrect.
AutoML provides a strong starting point, but automatically deploying a model without aligning it with business-specific requirements and metrics can lead to suboptimal outcomes.