NCA-GENL Question 177
Select 3You are managing a pipeline for training a large language model (LLM). During the data collection phase, you notice that the data ingestion process has slowed significantly, causing delays in your training schedule. Which of the following actions are appropriate to monitor and resolve the issue?
- A
Check the resource utilization (e.g., CPU, GPU, memory, and storage) on the data ingestion servers.
- B
Review the data preprocessing logs for errors or bottlenecks in pipeline stages.
- C
Immediately terminate the data ingestion process to prevent further delays.
- D
Set up automated alerts to monitor data throughput and flag anomalies in real time.
- E
Ignore the issue as the training process can proceed with partial data.
Show answer and explanation
Correct answers: A, B, D
Explanation
To effectively monitor and resolve issues in the data collection process, it is crucial to identify root causes through resource monitoring and log analysis. Automated alerts are also valuable for proactive detection and resolution of similar problems. Terminating the process prematurely or ignoring the issue can lead to data loss or suboptimal model performance, which should be avoided.
- A. Correct.
Monitoring resource utilization can help identify if hardware constraints (like network bandwidth or memory) are causing the slowdown.
- B. Correct.
Reviewing logs can help detect specific errors or bottlenecks in the data ingestion or preprocessing stages.
- C. Incorrect.
Terminating the data ingestion process without identifying the root cause can lead to data loss and worsen the situation.
- D. Correct.
Setting up automated alerts helps in proactive monitoring and ensures that similar issues are flagged and resolved promptly in the future.
- E. Incorrect.
Ignoring the issue is not advisable since training with incomplete or insufficient data can degrade model performance.