NCA-GENL Question 89
Single answerYou are tasked with evaluating the performance of two large language models (LLMs) trained on the same dataset. Model A has a lower mean squared error (MSE) but a slightly lower coefficient of determination (R²) compared to Model B. Which model would generally be considered better, and why?
- A
Model A, because a lower MSE indicates better prediction accuracy.
- B
Model B, because a higher R² indicates that it explains more variance in the data.
- C
Model A, because MSE is a more robust metric for LLM evaluation.
- D
Model B, because R² is always a more reliable performance metric than MSE.
Show answer and explanation
Correct answer: A
Explanation
When comparing models, MSE and R² provide complementary insights. MSE measures the average squared prediction error, making it directly indicative of model accuracy. R² measures the proportion of variance explained, which is useful but may not fully capture prediction accuracy. In this scenario, Model A is better because a lower MSE indicates more accurate predictions, which is typically prioritized when evaluating LLMs.
- A. Correct.
Correct. A lower MSE directly indicates better prediction accuracy, as it measures the average squared difference between predicted and actual values. While R² is useful, MSE is often the primary metric for evaluating regression-based models.
- B. Incorrect.
Incorrect. While a higher R² indicates a better proportion of explained variance, it does not necessarily mean the model has lower prediction error. MSE is often prioritized for evaluation.
- C. Incorrect.
Incorrect. MSE is a key metric but not inherently more 'robust' than R². The choice depends on the context, but in this case, lower MSE is the preferred indicator of performance.
- D. Incorrect.
Incorrect. R² is not always more reliable than MSE. It provides useful insights into how well the model explains variance, but it does not directly measure prediction error.