NCP-AII Question 8
Select 3An AI infrastructure team is running a deep learning workload on a multi-GPU system. During training, they notice significantly reduced performance and intermittent crashes. After checking the system logs, they observe frequent GPU 'Thermal Throttling' warnings. What could be the primary causes of this issue, and what actions should they take to resolve it?
- A
Check if the GPU fans are operational and clean any dust blocking airflow.
- B
Inspect the GPU utilization metrics to determine if the workload is too demanding for the hardware.
- C
Ensure the system's airflow is not obstructed and the data center's ambient temperature is within operational limits.
- D
Replace the network card as it might be causing the crashes.
- E
Run a GPU diagnostics tool to check for hardware faults or overheating issues.
Show answer and explanation
Correct answers: A, C, E
Explanation
Thermal throttling occurs when a GPU exceeds its safe operating temperature, causing it to reduce performance to prevent damage. Common causes include blocked airflow, dust buildup, or malfunctioning fans. Proper diagnostics and maintaining the system's cooling infrastructure are critical for resolving such issues and ensuring optimal performance.
- A. Correct.
Correct: Dust or non-functional GPU fans can result in poor cooling performance, leading to thermal throttling. Cleaning or repairing the fans can resolve this issue.
- B. Incorrect.
Incorrect: While GPU utilization metrics are useful for monitoring workload efficiency, high utilization alone does not directly cause thermal throttling or crashes.
- C. Correct.
Correct: Poor airflow or high ambient temperature can lead to increased GPU temperatures. Ensuring proper airflow and cooling will help prevent overheating.
- D. Incorrect.
Incorrect: A faulty network card is unlikely to cause GPU-specific thermal throttling or crashes; this option is irrelevant to the issue described.
- E. Correct.
Correct: Running a GPU diagnostics tool can help identify overheating or other hardware faults, making it an essential step in troubleshooting.