DEA-C01 Question 330
Select 3A data engineering team is designing a data processing pipeline on AWS to analyze streaming data from IoT devices. The team is deciding between using Amazon Kinesis Data Streams (provisioned) or AWS Lambda (serverless) for their compute layer. They need to minimize operational overhead while ensuring scalability and predictable performance for handling high-throughput workloads. Which factors should the team consider when making this decision?
- A
AWS Lambda eliminates the need to manage infrastructure but may face invocation limits for high-throughput workloads.
- B
Amazon Kinesis Data Streams requires capacity provisioning but provides more control over performance tuning.
- C
AWS Lambda is better for workloads requiring predictable and consistent performance under heavy throughput.
- D
Amazon Kinesis Data Streams has automatic scaling and does not require capacity planning.
- E
AWS Lambda is cost-efficient for sporadic workloads but may become expensive for sustained high-volume data processing.
Show answer and explanation
Correct answers: A, B, E
Explanation
When deciding between provisioned services like Amazon Kinesis Data Streams and serverless services like AWS Lambda, the team must weigh operational overhead, scalability, cost, and performance. AWS Lambda is serverless, reducing operational overhead, but it has limits that can affect high-throughput workloads and may become expensive for sustained processing. Amazon Kinesis Data Streams, on the other hand, requires capacity provisioning but offers more control over performance tuning and predictable performance, making it ideal for high-throughput and consistent workloads.
- A. Correct.
Correct. AWS Lambda is serverless and eliminates infrastructure management. However, it has soft and hard limits, such as concurrency limits, which can impact its ability to handle high-throughput workloads.
- B. Correct.
Correct. Kinesis Data Streams requires explicit capacity provisioning (shard management), which allows for fine-grained control of performance and scalability.
- C. Incorrect.
Incorrect. AWS Lambda does not offer predictable performance under heavy throughput due to potential throttling and invocation limits, which can impact performance consistency.
- D. Incorrect.
Incorrect. While Kinesis Data Streams is scalable, it does not automatically scale. Capacity provisioning is required to handle increased throughput by adding shards.
- E. Correct.
Correct. AWS Lambda is cost-efficient for intermittent workloads due to its pay-per-use model but can become costly under sustained high-volume traffic as the number of invocations increases.