NCA-GENL Question 104
Select 3You are tasked with analyzing the performance of a generative AI model designed to summarize lengthy legal documents. During your analysis, you discover that the model's accuracy varies significantly depending on the length and complexity of the input text. Which factors should you consider as most likely to affect the results of this research?
- A
The distribution of document lengths in the training dataset
- B
The type and quality of legal documents used for testing
- C
The hardware specifications of the GPU used for training
- D
The token limit imposed by the language model
- E
The licensing terms of the dataset used for training
Show answer and explanation
Correct answers: A, B, D
Explanation
To identify relationships and trends in generative AI research, it is essential to consider factors like data distribution, testing dataset quality, and model constraints (e.g., token limits). These directly influence the model's ability to generalize and perform accurately across varying inputs. Hardware specifications and licensing terms, while important for other aspects of the project, do not affect the observed trends or outcomes of the research itself.
- A. Correct.
The distribution of document lengths in the training dataset is crucial because if the training data does not adequately represent longer or more complex texts, the model may struggle with those types of inputs.
- B. Correct.
The type and quality of legal documents used for testing directly impact the evaluation. If the testing documents differ significantly in style, complexity, or structure from the training data, it could lead to inaccurate results.
- C. Incorrect.
The hardware specifications of the GPU used for training are important for performance and speed but do not directly affect the relationships or trends in the research results.
- D. Correct.
The token limit imposed by the language model is critical because exceeding this limit can truncate inputs, leading to incomplete or inaccurate summaries.
- E. Incorrect.
The licensing terms of the dataset used for training are relevant for legal compliance but do not influence the model's performance or observed trends in research results.