NCA-GENL Question 125
Single answerYou are comparing the performance of two large language models (LLMs) trained on the same dataset. Model A has a lower mean squared error (MSE) on the validation set compared to Model B, but Model B has a higher R² score. Which model would be considered better if the focus is on explaining the variance in the data?
- A
Model A, because a lower MSE indicates better performance.
- B
Model B, because a higher R² score indicates better explained variance.
- C
Model A, as MSE is always a more reliable metric than R².
- D
Model B, as R² can directly measure how well the model captures the variability in the data.
Show answer and explanation
Correct answer: B
Explanation
When comparing models, the choice of metric depends on the specific goal. In this case, since the focus is on explaining the variability in the data, R² is the more relevant metric. A higher R² score for Model B indicates it better explains the variance, making it the preferred choice over Model A.
- A. Incorrect.
Model A may have a lower MSE, but MSE does not directly reflect how well the model captures the variance in the data. It measures the average squared difference between predictions and actual values.
- B. Correct.
Correct. A higher R² score indicates that Model B better explains the proportion of variance in the data, which aligns with the focus of this scenario.
- C. Incorrect.
This is incorrect. While MSE is a useful metric, R² provides a direct measure of explained variance, which is the focus here.
- D. Incorrect.
Although R² measures explained variance, this option is redundant because the correct explanation is already captured in the second option.