NCA-GENL Question 68
Select 3You are tasked with preparing a dataset for training a large language model (LLM). During the data preprocessing stage, you notice that the dataset contains duplicate entries, missing values, and inconsistent formatting. Which steps should you prioritize to ensure the data is optimized for model training?
- A
Remove duplicate entries to avoid redundant data during training.
- B
Impute missing values with random numbers to fill gaps in the dataset.
- C
Normalize text formatting, such as converting all text to lowercase and removing special characters.
- D
Add synthetic data to increase the size of the dataset without addressing existing issues.
- E
Perform exploratory data analysis (EDA) to identify patterns and data quality issues.
Show answer and explanation
Correct answers: A, C, E
Explanation
To prepare data for training an LLM, it is essential to focus on cleaning and transforming the data to ensure consistency, quality, and relevance. Removing duplicates, normalizing text formatting, and performing EDA are standard and effective practices for identifying and addressing data quality issues. These steps prevent redundancy, maintain consistency, and provide insights into the dataset, which are critical for successful model training.
- A. Correct.
Removing duplicate entries is crucial to avoid redundancy, which can negatively impact model training and lead to overfitting on repeated data.
- B. Incorrect.
Imputing missing values with random numbers is not a recommended practice, as it introduces noise and can degrade training performance.
- C. Correct.
Normalizing text formatting ensures consistency in the input data, which is especially important for tokenization and downstream processing in LLMs.
- D. Incorrect.
Adding synthetic data without addressing existing issues does not solve fundamental data quality problems and may propagate errors into the model.
- E. Correct.
Performing exploratory data analysis (EDA) helps uncover patterns, inconsistencies, and gaps in the data, enabling informed decisions during preprocessing.