DEA-C01 Question 221
Select 3A company is using AWS Glue for an ETL (Extract, Transform, Load) process to transform raw data from an Amazon S3 bucket into structured data stored in Amazon Redshift. The company notices that the ETL job fails intermittently due to an 'OutOfMemoryError'. Which of the following actions should the data engineering team take to resolve this issue?
- A
Increase the Worker Type in the AWS Glue job to G.2X or G.4X.
- B
Enable the 'Job Bookmark' feature in AWS Glue to avoid reprocessing data.
- C
Split the input data into smaller partitions using AWS Glue DynamicFrames before processing.
- D
Increase the number of DPU (Data Processing Units) allocated to the Glue job.
- E
Enable S3 Transfer Acceleration to improve data transfer speed.
Show answer and explanation
Correct answers: A, C, D
Explanation
The 'OutOfMemoryError' in AWS Glue jobs is typically caused by insufficient memory or processing power. Actions such as increasing the Worker Type, splitting input data into smaller partitions, and allocating more DPUs directly address this issue. The 'Job Bookmark' feature and S3 Transfer Acceleration are unrelated to memory management and do not resolve the error in this scenario.
- A. Correct.
Increasing the Worker Type to G.2X or G.4X provides more memory and CPUs for the Glue job, which can help prevent 'OutOfMemoryError' issues during data processing.
- B. Incorrect.
While the 'Job Bookmark' feature helps avoid reprocessing already processed data, it does not address memory-related issues in the Glue job.
- C. Correct.
Splitting the input data into smaller partitions using Glue DynamicFrames can reduce the memory load on individual worker nodes, helping to avoid memory-related errors.
- D. Correct.
Increasing the number of DPUs allocated to the Glue job provides more processing power and memory, which can help address out-of-memory errors.
- E. Incorrect.
S3 Transfer Acceleration is used for faster data transfer to and from S3 but does not directly resolve memory-related issues in AWS Glue jobs.