NCA-AIIO Question 103
Select 3You are tasked with deploying an AI workload on an NVIDIA-powered cluster that uses Kubernetes for orchestration. The workload involves training a deep learning model requiring GPUs and has strict deadline requirements. Which of the following practices should you implement for efficient job scheduling and resource allocation?
- A
Use GPU node labels and taints to ensure that only GPU-required jobs are scheduled on GPU-enabled nodes.
- B
Enable preemption of lower-priority jobs to prioritize the deep learning workload within the cluster.
- C
Configure horizontal pod autoscaling to dynamically scale the number of GPUs allocated to the job based on its resource usage.
- D
Use gang scheduling to ensure all required GPUs for the job are allocated simultaneously before starting the workload.
- E
Deploy the workload on a single node to avoid potential communication overhead between nodes.
Show answer and explanation
Correct answers: A, B, D
Explanation
Efficient job scheduling and resource allocation are critical for AI workloads in GPU-accelerated clusters. Practices such as using GPU node labels and taints, enabling job preemption for priority tasks, and leveraging gang scheduling help optimize resource use and meet workload deadlines. Other approaches like horizontal pod autoscaling or limiting workloads to a single node may not be suitable for GPU-optimized AI clusters.
- A. Correct.
Using GPU node labels and taints helps ensure that GPU resources are allocated only to jobs that require them, avoiding resource contention and improving efficiency.
- B. Correct.
Enabling job preemption allows higher-priority jobs, such as those with strict deadlines, to take precedence over lower-priority jobs, ensuring timely execution.
- C. Incorrect.
Horizontal pod autoscaling adjusts the number of pods, not GPUs specifically. While useful for CPU-based workloads, it does not directly manage GPU resources.
- D. Correct.
Gang scheduling ensures that all resources required by a job (such as multiple GPUs) are allocated simultaneously, preventing resource fragmentation and delays in job execution.
- E. Incorrect.
Deploying a workload on a single node can limit scalability and introduce bottlenecks, especially for high-performance computing tasks that benefit from distributed execution.