DEA-C01 Question 511
Select 3A company is using AWS Glue to process large volumes of data stored in Amazon S3 and load it into an Amazon Redshift cluster. They want to optimize the performance of their Glue ETL jobs and reduce costs. Which of the following actions should they take?
- A
Enable job bookmarking in AWS Glue to avoid reprocessing data that has already been processed.
- B
Use Amazon Redshift Spectrum to query the data in S3 directly, bypassing the need for AWS Glue ETL jobs.
- C
Configure AWS Glue to use worker types like G.1X or G.2X to match the job's computational requirements.
- D
Partition the data in Amazon S3 by commonly queried fields to optimize Glue job performance.
- E
Use AWS Glue's built-in transformations exclusively, avoiding custom transformations in PySpark or Scala.
Show answer and explanation
Correct answers: A, C, D
Explanation
To optimize AWS Glue ETL jobs, enabling job bookmarking can reduce unnecessary data reprocessing, while configuring the correct worker types ensures that computational resources are used efficiently. Partitioning data in Amazon S3 enhances job performance by minimizing data scanned. These strategies collectively improve performance and reduce costs. Redshift Spectrum, while a valid querying approach, is not relevant to optimizing Glue ETL jobs, and avoiding custom transformations entirely may not be feasible for complex use cases.
- A. Correct.
Enabling job bookmarking ensures that AWS Glue processes only new or updated data, reducing unnecessary reprocessing and costs.
- B. Incorrect.
While Amazon Redshift Spectrum can query data directly in S3, this approach bypasses the Glue ETL process, which is central to this use case.
- C. Correct.
Choosing appropriate worker types in AWS Glue ensures that the computing resources are optimized for the job's requirements, improving performance and reducing costs.
- D. Correct.
Partitioning data in Amazon S3 improves Glue job performance by allowing it to process only the relevant subsets of data, reducing the amount of data scanned and processed.
- E. Incorrect.
While AWS Glue's built-in transformations are efficient, custom transformations in PySpark or Scala are sometimes necessary for advanced use cases and cannot be entirely avoided.