NCA-GENL Question 103
Select 3You are training a Large Language Model (LLM) on a dataset for sentiment analysis. After analyzing the training results, you notice that the model's accuracy is lower on reviews from a specific product category. Which factors should you consider to identify relationships or trends that might explain this performance issue?
- A
The distribution of product categories in the training dataset
- B
The tokenization method used to preprocess the text data
- C
The number of layers in the LLM architecture
- D
The presence of biased or unbalanced sentiment labels in the specific product category
- E
The size of the training dataset relative to the model's parameter count
Show answer and explanation
Correct answers: A, B, D
Explanation
To identify relationships and trends affecting the model's performance, it is essential to analyze the representation of product categories in the dataset, preprocessing methods like tokenization, and the balance of sentiment labels in the underperforming category. These factors directly influence the model's ability to learn and generalize patterns specific to the product category. Other aspects, such as model layers or dataset size relative to parameter count, are more generalized considerations and do not directly address the issue at hand.
- A. Correct.
If the product category is underrepresented in the training dataset, the model may perform poorly on it due to insufficient exposure during training. This relationship is critical to analyze.
- B. Correct.
Tokenization methods impact how the text is converted into inputs for the model. Poor tokenization might fail to capture important patterns in the specific product category, affecting performance.
- C. Incorrect.
While the number of layers in the LLM architecture affects overall model capacity, it is not directly relevant to identifying relationships or trends specific to an underperforming product category.
- D. Correct.
Biased or unbalanced sentiment labels in the specific product category could skew the model's learning process, impacting its ability to generalize correctly in that category.
- E. Incorrect.
While dataset size relative to model parameters is important for general training stability, it is not directly related to identifying category-specific performance issues.