DEA-C01 Question 329
Select 3A data engineering team is designing a data processing pipeline for analyzing customer transaction data. The pipeline is expected to handle highly variable traffic, with significant traffic spikes during sales events. The team is considering whether to use AWS Glue (a serverless service) or an Amazon EMR cluster (a provisioned service) for their data transformations. Which considerations should influence their decision?
- A
Amazon EMR allows fine-grained control over the underlying infrastructure, which is beneficial if specific instance types or custom configurations are required.
- B
AWS Glue is a cost-effective option for intermittent workloads due to its pay-as-you-go pricing model.
- C
Amazon EMR provides automatic scaling out of the box, which makes it ideal for handling unpredictable traffic spikes.
- D
AWS Glue has built-in integrations with AWS services and provides a serverless architecture, reducing the operational overhead.
- E
Amazon EMR is always more cost-effective than AWS Glue for high throughput workloads, regardless of workload patterns.
Show answer and explanation
Correct answers: A, B, D
Explanation
When choosing between provisioned services like Amazon EMR and serverless services like AWS Glue, it is important to consider workload patterns, cost implications, and operational requirements. AWS Glue is a good choice for intermittent, variable workloads due to its serverless architecture and pay-as-you-go pricing. Amazon EMR provides greater control over infrastructure and can be optimized for specific workloads, but it involves more operational overhead and may not be as cost-effective for workloads with irregular traffic patterns.
- A. Correct.
Correct: Amazon EMR provides full control over the underlying infrastructure, including instance types, cluster size, and custom configurations. This is beneficial for workloads requiring specific hardware or software setups.
- B. Correct.
Correct: AWS Glue uses a serverless, pay-as-you-go pricing model, which makes it cost-effective for workloads that do not require constant resource utilization.
- C. Incorrect.
Incorrect: Amazon EMR does not provide automatic scaling out of the box. While it supports auto-scaling, it requires configuration and does not scale as seamlessly as serverless services like AWS Glue.
- D. Correct.
Correct: AWS Glue is serverless and reduces the operational overhead of managing infrastructure. Its integrations with other AWS services, such as S3 and Athena, simplify data workflows.
- E. Incorrect.
Incorrect: While Amazon EMR can be cost-effective for consistent, high throughput workloads, it is not always more cost-effective than AWS Glue. The choice depends on the specific workload patterns and resource usage.