MLA-C01 Question 218
Select 3You are building a machine learning model to classify customer reviews as positive or negative. Before deploying the model, you want to establish a performance baseline to measure the effectiveness of your solution. Which of the following methods can you use to create a reliable performance baseline?
- A
Use a simple heuristic, such as classifying all reviews as the majority class.
- B
Train a basic machine learning model, such as logistic regression, and evaluate its performance.
- C
Deploy the model directly in production and collect feedback from users to determine its performance.
- D
Evaluate the model's performance against a random classifier.
- E
Manually label a subset of the data and calculate performance metrics using human annotations.
Show answer and explanation
Correct answers: A, B, E
Explanation
Establishing a performance baseline is critical for evaluating the effectiveness of a machine learning model. Methods such as using simple heuristics, training basic models, and leveraging human annotations provide meaningful benchmarks for comparison. These methods allow you to determine if your model is actually improving upon naive or simpler approaches, ensuring a robust evaluation before deployment.
- A. Correct.
Using a simple heuristic, such as classifying all reviews as the majority class, provides a baseline for comparison. This can help identify whether your model performs better than naive assumptions.
- B. Correct.
Training a basic machine learning model, such as logistic regression, is a common approach to establish a baseline. It allows you to compare the performance of your advanced model against a simpler one.
- C. Incorrect.
Deploying the model in production without prior evaluation is not a recommended practice for establishing a baseline, as it introduces risks and lacks a controlled performance benchmark.
- D. Incorrect.
A random classifier can provide a performance benchmark, but it is generally not considered a meaningful or reliable baseline for most practical machine learning tasks.
- E. Correct.
Manually labeling a subset of data and calculating performance metrics based on human annotations is a valid and reliable way to establish a performance baseline, as it reflects ground truth.