NCA-GENL Question 139
Select 3You are training a Large Language Model (LLM) for a customer support chatbot. During your research, you notice that the chatbot's performance varies significantly across different user demographics. Which factors should you investigate to identify relationships or trends that might be affecting the model's results?
- A
The distribution of training data across different demographic groups
- B
The hardware specifications of the machines used to train the model
- C
Biases in the labeling process of the training dataset
- D
The frequency of updates to the LLM's parameters during training
- E
The specific prompts used to evaluate the chatbot's performance
Show answer and explanation
Correct answers: A, C, E
Explanation
To identify relationships and trends affecting the model's performance, it's essential to examine factors directly tied to the data and evaluation process, such as data distribution, labeling biases, and prompt design. These factors can introduce systematic biases and impact the LLM's ability to perform effectively across demographics, while unrelated aspects such as hardware specifications or training update frequency do not directly influence demographic-specific results.
- A. Correct.
The distribution of training data across different demographic groups is critical to investigate, as an imbalance may lead to biased or ineffective results for underrepresented groups.
- B. Incorrect.
While hardware specifications can influence training speed and efficiency, they do not directly affect the relationships or trends in the model's performance across demographics.
- C. Correct.
Biases in the labeling process can introduce systematic errors into the training dataset, affecting the model's ability to generalize fairly across demographics.
- D. Incorrect.
The frequency of parameter updates may affect model convergence but does not directly explain performance issues related to specific user demographics.
- E. Correct.
The specific prompts used during evaluation can reveal performance trends and biases, as poorly designed prompts may not adequately test the model's robustness across different groups.