MLA-C01 Question 390
Select 3You are managing an AWS SageMaker training job for a deep learning model that processes large amounts of image data. The job uses a GPU instance type for accelerated training. After reviewing your AWS billing dashboard, you notice higher-than-expected costs attributed to the training job. What actions can you take to monitor and optimize the infrastructure and reduce costs effectively?
- A
Enable Amazon CloudWatch Metrics and set up alarms to monitor GPU utilization during the training job.
- B
Switch the instance type to a CPU-based instance to reduce costs, even if it increases training time.
- C
Use Amazon SageMaker Debugger to profile the training job and identify idle resources or bottlenecks.
- D
Leverage SageMaker Managed Spot Training to utilize spot instances for reduced costs.
- E
Pre-process the image data locally to reduce the size of the dataset before uploading it to SageMaker.
Show answer and explanation
Correct answers: A, C, D
Explanation
To monitor and optimize infrastructure and reduce costs for a SageMaker training job, it is crucial to focus on resource utilization and leveraging cost-efficient features. CloudWatch Metrics allows real-time monitoring of GPU usage, while SageMaker Debugger helps identify inefficiencies in the training process. SageMaker Managed Spot Training significantly reduces costs by using spot instances. Together, these approaches effectively manage and optimize your training job's infrastructure and cost.
- A. Correct.
Correct. Enabling CloudWatch Metrics allows you to monitor resource utilization like GPU usage. Setting alarms helps you detect underutilized resources and take corrective actions to optimize costs.
- B. Incorrect.
Incorrect. While switching to a CPU instance may reduce costs, it is not practical for deep learning tasks that require GPUs for performance. This could drastically increase training time and negate any cost savings.
- C. Correct.
Correct. SageMaker Debugger provides profiling tools to identify inefficiencies in the training job, such as idle GPU time or data bottlenecks, which can help optimize resource usage and reduce costs.
- D. Correct.
Correct. SageMaker Managed Spot Training allows you to use spot instances, which are significantly cheaper than on-demand instances, to run your training jobs with minimal disruption.
- E. Incorrect.
Incorrect. Pre-processing data locally may reduce data upload costs, but it does not directly address the cost of the SageMaker training job itself. Optimization efforts should focus on the training infrastructure and execution.