NCA-GENL Question 87
Single answerYou are tasked with evaluating two large language models (LLMs) trained for text summarization tasks. Model A has a Mean Squared Error (MSE) of 0.12 and an R-squared (R²) value of 0.85, while Model B has an MSE of 0.08 and an R² value of 0.65. Which model would you select if your primary goal is to achieve the most accurate predictions that align well with the variability of the data?
- A
Model A, because it has a higher R-squared value indicating better explained variance.
- B
Model B, because it has a lower Mean Squared Error indicating better prediction accuracy.
- C
Model A, because it balances both MSE and R-squared effectively.
- D
Model B, because R-squared is less important than MSE for evaluating prediction accuracy.
Show answer and explanation
Correct answer: C
Explanation
Both MSE and R-squared are critical statistical performance metrics when comparing models. MSE measures prediction accuracy, while R-squared evaluates how well the model explains the variability in the data. In this scenario, Model A has a slightly higher MSE but a superior R-squared, making it the better choice for a balanced evaluation of accuracy and data alignment.
- A. Incorrect.
While Model A has a higher R-squared value, focusing only on explained variance without considering prediction accuracy may lead to suboptimal decisions.
- B. Incorrect.
Although Model B has a lower Mean Squared Error, the significantly lower R-squared value indicates it does not align well with the variability of the data.
- C. Correct.
Model A strikes a balance between a relatively low MSE and high R-squared, providing both accurate predictions and good alignment with the data's variability.
- D. Incorrect.
While MSE is important for prediction accuracy, disregarding R-squared altogether can result in a model that poorly represents the underlying data patterns.