MLA-C01 Question 216
Select 3You are working on a machine learning project to predict customer churn for a subscription-based service. To evaluate the performance of your model, you want to establish a performance baseline. Which of the following methods can you use to create a robust baseline for comparison?
- A
Use a simple heuristic model, such as predicting 'no churn' for all customers, and calculate its performance metrics.
- B
Train a basic machine learning model, such as logistic regression, and measure its performance.
- C
Directly deploy the initial machine learning model into production without analysis, as baselines are not required for deployment.
- D
Use historical data to calculate the accuracy of random guessing and set it as the baseline.
- E
Perform hyperparameter tuning on the model to determine the best performance and set it as the baseline.
Show answer and explanation
Correct answers: A, B, D
Explanation
Establishing a performance baseline is essential for evaluating the effectiveness of a machine learning model. Common methods include using simple heuristic approaches, basic machine learning models, and random guessing based on historical data. These baselines provide benchmarks to determine whether the final model's performance is meaningful and improved. Skipping baseline creation or conflating it with optimization steps, such as hyperparameter tuning, is not aligned with best practices.
- A. Correct.
Using a simple heuristic model, such as predicting 'no churn' for all customers, is a common and effective method for establishing a baseline. It provides a point of comparison to determine whether your machine learning model performs better than a basic, naive approach.
- B. Correct.
Training a basic machine learning model, such as logistic regression, provides a more advanced baseline for comparison. It helps you understand if the added complexity of your final model improves performance.
- C. Incorrect.
Skipping baseline creation entirely and directly deploying the model is not recommended. Baselines are critical for comparing model performance and ensuring the model adds value.
- D. Correct.
Using historical data to calculate the accuracy of random guessing is another valid method for creating a baseline. It helps provide a lower boundary for performance expectations.
- E. Incorrect.
Performing hyperparameter tuning is aimed at optimizing a model's performance, not establishing a baseline. Baselines are typically created before tuning begins to provide a point of reference.