AIF-C01 Question 137
Select 3Your company is planning to fine-tune a foundation model for a specific use case involving sentiment analysis of customer reviews. Which of the following steps should be included in the fine-tuning process to ensure optimal performance?
- A
Collect a labeled dataset that is representative of the target domain.
- B
Replace the foundation model's architecture entirely with a custom-built model.
- C
Freeze most of the foundation model's layers and only train the final layers specific to the task.
- D
Select a learning rate suitable for fine-tuning rather than training from scratch.
- E
Use a foundation model pre-trained on a completely unrelated domain to save time.
Show answer and explanation
Correct answers: A, C, D
Explanation
Fine-tuning a foundation model involves adapting a pre-trained model to a specific use case. This process includes preparing a labeled dataset that is closely aligned with the target task, freezing most of the pre-trained layers to retain general knowledge, and fine-tuning the few task-specific layers with a suitable learning rate. Using a model pre-trained on a related domain ensures better transfer learning, whereas replacing the architecture or relying on unrelated pre-trained models is not recommended.
- A. Correct.
Collecting a labeled dataset that reflects the target domain is critical for fine-tuning because the foundation model needs to align with the specific task and data characteristics.
- B. Incorrect.
Replacing the foundation model's architecture defeats the purpose of using a foundation model, as it eliminates its pre-trained knowledge.
- C. Correct.
Freezing most of the foundation model's layers and training only the task-specific layers is a common approach in fine-tuning, as it retains the general knowledge while adapting to the specific task.
- D. Correct.
Choosing an appropriate learning rate for fine-tuning ensures the model updates weights incrementally without overwriting the pre-trained knowledge.
- E. Incorrect.
Using a foundation model pre-trained on an unrelated domain may lead to suboptimal results because the model's initial knowledge may not generalize to the target task effectively.