NCA-GENL Question 69
Select 3You are preparing a dataset for training a large language model (LLM). The dataset contains missing values, duplicate entries, and inconsistent text formatting (e.g., mixed capitalization). Which of the following steps are essential to ensure the data quality before training the LLM?
- A
Remove or impute missing values in the dataset.
- B
Normalize text formatting to ensure consistency (e.g., converting all text to lowercase).
- C
Keep duplicate records to augment the dataset size.
- D
Perform tokenization and ensure that the vocabulary aligns with the LLM's requirements.
- E
Randomly delete a portion of the data to reduce preprocessing time.
Show answer and explanation
Correct answers: A, B, D
Explanation
Inspecting, cleansing, and transforming data are critical steps in preparing datasets for training LLMs. The correct steps, such as handling missing values, normalizing text, and tokenizing data, ensure that the input is clean and compatible with the model, which directly impacts the model's performance. Incorrect practices, like retaining duplicates or randomly deleting data, can degrade the quality of the dataset and negatively affect the results.
- A. Correct.
Removing or imputing missing values helps ensure that the dataset does not introduce noise or errors during model training.
- B. Correct.
Normalizing text formatting ensures that the model learns patterns in the data without being confused by inconsistencies, such as mixed capitalization.
- C. Incorrect.
Keeping duplicate records can introduce bias and redundancy in the training process, which is not advisable for high-quality model performance.
- D. Correct.
Performing tokenization and ensuring vocabulary alignment with the LLM's requirements are critical preprocessing steps to make the dataset compatible with the model.
- E. Incorrect.
Randomly deleting portions of data can lead to loss of valuable information and is generally not a recommended practice for improving dataset quality.