Databricks Generative AI Engineer Associate Question 37
Select 3You are tasked with preparing a dataset to fine-tune a large language model (LLM) for a text summarization task. The dataset contains raw user-generated content with inconsistent formatting, noise (e.g., HTML tags), and missing values. Which of the following steps would be most appropriate to prepare the dataset for fine-tuning?
- A
Remove HTML tags and other noise using text cleaning techniques.
- B
Replace all missing values with the string 'UNKNOWN'.
- C
Normalize the text by converting it to lowercase and removing extra whitespace.
- D
Perform tokenization and padding to ensure input sequences are of uniform length.
- E
Directly feed the raw dataset into the model without modification.
Show answer and explanation
Correct answers: A, C, D
Explanation
Data preparation is a critical step in fine-tuning a large language model. Cleaning the text (e.g., removing HTML tags), normalizing it, and ensuring uniform input lengths through tokenization and padding are essential tasks to ensure the model can learn effectively. Ignoring these steps or introducing unnecessary noise, such as replacing missing values with placeholder strings, can significantly degrade model performance.
- A. Correct.
Removing HTML tags and other noise is critical since LLMs perform better on clean and structured data. This step ensures the model learns from relevant information.
- B. Incorrect.
Replacing missing values with 'UNKNOWN' may introduce noise or irrelevant tokens that could confuse the model during training. This is not a recommended practice.
- C. Correct.
Text normalization, such as converting to lowercase and removing extra whitespace, ensures uniformity in input data, which is key to improving model performance.
- D. Correct.
Tokenization and padding are necessary for LLMs to process text inputs efficiently, as models typically expect inputs to have uniform lengths.
- E. Incorrect.
Feeding raw data into the model is not advisable as it can contain noise, inconsistencies, and unstructured text, leading to poor model performance.