Google Professional Machine Learning Engineer Question 276
Select 4Google Cloud PlatformYou are tasked with fine-tuning a foundational model for a text classification task using Vertex AI Model Garden. The dataset you have is relatively small but labeled, and the foundational model you are using was pre-trained on a large corpus of general text data. What steps should you take to fine-tune the foundational model efficiently while preventing overfitting?
- A
Use transfer learning to leverage the pre-trained model and only train the final classification layer with your labeled dataset.
- B
Freeze the majority of the foundational model’s layers during fine-tuning to retain pre-trained knowledge.
- C
Train the entire foundational model from scratch using your labeled dataset to ensure it learns task-specific features.
- D
Enable learning rate scheduling to dynamically adjust the learning rate during fine-tuning.
- E
Apply data augmentation techniques to synthetically expand the labeled dataset before fine-tuning.
Show answer and explanation
Correct answers: A, B, D, E
Explanation
Fine-tuning a foundational model with a small labeled dataset requires careful steps to leverage the pre-trained knowledge while avoiding overfitting. Using transfer learning, freezing the majority of the layers, and enabling learning rate scheduling are effective practices. Additionally, data augmentation can help improve the diversity of the dataset. Training the entire model from scratch is not recommended in scenarios with limited data.
- A. Correct.
Correct. Transfer learning allows you to leverage the pre-trained features of the foundational model and only fine-tune the final layers, making it efficient for small datasets.
- B. Correct.
Correct. Freezing most pre-trained layers helps prevent overfitting and retains the general knowledge of the foundational model while fine-tuning task-specific layers.
- C. Incorrect.
Incorrect. Training the entire foundational model from scratch with a small dataset will likely lead to overfitting and is not efficient.
- D. Correct.
Correct. Learning rate scheduling helps optimize training by adjusting the learning rate dynamically, often leading to better convergence during fine-tuning.
- E. Correct.
Correct. Data augmentation can help expand the small dataset, providing the model with more varied examples and reducing the risk of overfitting.