Databricks Generative AI Engineer Associate Question 41
Select 3You are working with a dataset containing text and numerical features to train a generative AI model in Databricks. Before training, you need to preprocess the data to ensure consistent input formats and handle any missing values. Which of the following steps are most appropriate for preparing the dataset for training in this scenario?
- A
Tokenize the text data and convert it into numerical representations such as embeddings or one-hot encodings.
- B
Remove rows with missing numerical values without considering their impact on the dataset.
- C
Normalize numerical features to a consistent scale, such as using Min-Max scaling or Standardization.
- D
Apply data augmentation techniques to generate additional synthetic numerical data.
- E
Fill missing numerical values using strategies like mean, median, or a predictive model.
Show answer and explanation
Correct answers: A, C, E
Explanation
Preparing a dataset for training involves converting text data into numerical formats so models can process it effectively, handling missing values to avoid data loss, and normalizing numerical features to ensure consistent scaling. These steps are critical for creating a balanced and well-prepared dataset.
- A. Correct.
Correct: Tokenizing and converting text data into numerical representations are essential steps for preparing text features for machine learning models, as models cannot process raw text directly.
- B. Incorrect.
Incorrect: Simply removing rows with missing values can lead to data loss, and this approach is not ideal without evaluating its impact on the dataset.
- C. Correct.
Correct: Normalizing numerical features ensures that all features contribute equally to the model training, avoiding bias due to large-scale differences.
- D. Incorrect.
Incorrect: Data augmentation is typically used for image or text data, not for numerical data, and it is not relevant in this scenario.
- E. Correct.
Correct: Filling missing numerical values with strategies like mean or median ensures the dataset remains consistent and avoids dropping rows unnecessarily.