NCA-GENL Question 124
Single answerYou are tasked with comparing the performance of two large language models (LLMs) designed for text summarization. Model A has a lower mean squared error (MSE) on the validation set compared to Model B, but Model B has a higher R² (coefficient of determination) value. Which of the following statements best explains how these metrics should influence your model selection?
- A
Model A is better because a lower MSE indicates it makes fewer errors on average.
- B
Model B is better because a higher R² indicates it explains more variance in the target data.
- C
Both models are equivalent in performance since different metrics are used.
- D
The choice of the best model depends on the specific task requirements and the importance of each metric.
Show answer and explanation
Correct answer: D
Explanation
MSE and R² measure different aspects of model performance. MSE measures the average squared error, indicating how close the predictions are to the target values. R² measures the proportion of variance in the target variable that the model explains. The choice of the best model depends on the specific requirements of the task, such as whether minimizing errors or capturing data variance is more critical.
- A. Incorrect.
While a lower MSE indicates that Model A has smaller average errors, it does not account for how well the model captures the overall variance in the data.
- B. Incorrect.
A higher R² for Model B suggests it explains more variance in the target data, but this alone does not provide a complete picture of model performance.
- C. Incorrect.
This is incorrect because the two models are not equivalent in performance; they perform differently based on the metrics used.
- D. Correct.
This is correct because the choice of the best model depends on the task's priorities. If minimizing individual errors is critical, MSE is more important, but if capturing the overall variance is the goal, R² might take precedence.