MLA-C01 Question 217
Select 3You are tasked with building a machine learning model to predict customer churn for a subscription-based service. Before deploying the model, you want to establish a performance baseline to evaluate the model's effectiveness. Which of the following methods can be used to create a reliable performance baseline?
- A
Use a simple statistical model such as predicting the majority class (e.g., customers will not churn).
- B
Train a complex deep learning model to establish a baseline with high performance.
- C
Use historical data to calculate metrics like churn rate and compare them against the model’s performance.
- D
Perform random guessing and compute its performance metrics.
- E
Evaluate the model directly on the test dataset without creating a baseline.
Show answer and explanation
Correct answers: A, C, D
Explanation
Creating performance baselines is a fundamental step in machine learning model development. Baselines, such as statistical heuristics, historical data metrics, or random guessing, provide reference points to determine if the model is adding meaningful predictive power. Avoid using overly complex models or skipping baselines entirely, as this can make it difficult to measure relative performance improvements.
- A. Correct.
Correct: A simple statistical model, such as predicting the majority class, is a common baseline to measure whether your machine learning model performs better than trivial heuristics.
- B. Incorrect.
Incorrect: Using a complex model for a baseline is not a good practice as baselines are typically simple and interpretable approaches to set a starting point for evaluation.
- C. Correct.
Correct: Historical data provides a reference point, allowing you to understand past trends and set expectations for your model's performance.
- D. Correct.
Correct: Random guessing is another simple baseline that helps assess whether the model is adding value beyond chance-level predictions.
- E. Incorrect.
Incorrect: Directly evaluating the model without any baseline does not provide a reference point to assess its performance. Baselines are critical to measure relative improvement.