DEA-C01 Question 49
Single answerYour organization is running a real-time analytics pipeline on AWS. Data is ingested into Amazon Kinesis Data Streams, processed by AWS Lambda, and stored in Amazon S3. The data engineering team needs to ensure the pipeline is fault-tolerant and can handle intermittent failures in the Lambda processing stage while ensuring data is not lost. What is the best approach to achieve this?
- A
Enable the retry feature in AWS Lambda and configure a Dead Letter Queue (DLQ) for failed events.
- B
Use Amazon Kinesis Data Analytics to process the data instead of AWS Lambda.
- C
Enable enhanced fan-out on Amazon Kinesis Data Streams to improve processing speeds.
- D
Increase the shard count in Amazon Kinesis Data Streams to ensure higher throughput.
Show answer and explanation
Correct answer: A
Explanation
To ensure fault tolerance in the real-time analytics pipeline, enabling retries in AWS Lambda allows transient errors to be retried automatically. Configuring a Dead Letter Queue (DLQ) ensures that events that fail after retries are captured and stored for further analysis, preventing data loss. This approach specifically addresses the requirement for handling intermittent failures in the Lambda processing stage.
- A. Correct.
Enabling retries in AWS Lambda ensures that transient failures are retried and configuring a Dead Letter Queue (DLQ) ensures that failed events are captured for further analysis, making the system fault-tolerant.
- B. Incorrect.
Using Amazon Kinesis Data Analytics is not directly relevant to fault tolerance for AWS Lambda. It is primarily used for real-time analytics, not for handling failures in processing pipelines.
- C. Incorrect.
Enhanced fan-out in Amazon Kinesis Data Streams improves throughput by allowing multiple consumers, but it does not address fault tolerance or handling of intermittent failures in AWS Lambda.
- D. Incorrect.
Increasing the shard count in Amazon Kinesis Data Streams improves throughput but does not handle the fault tolerance or failure scenarios in the Lambda processing stage.