DEA-C01 Question 289
Select 2You are a data engineer managing a data pipeline that ingests, processes, and stores streaming data using Amazon Kinesis Data Streams, AWS Lambda, and Amazon S3. Recently, you noticed that data is not being written to S3, and the Lambda function is reporting 'ProvisionedThroughputExceededException' errors. What steps should you take to resolve this issue?
- A
Increase the shard count in the Kinesis Data Stream.
- B
Optimize the Lambda function code to reduce execution time.
- C
Enable S3 bucket versioning to handle data overwrites.
- D
Implement retries with exponential backoff in the Lambda function.
- E
Increase the batch size in the Lambda function's event source mapping.
Show answer and explanation
Correct answers: A, D
Explanation
The 'ProvisionedThroughputExceededException' occurs when the data throughput for a Kinesis shard is exceeded. Increasing the shard count in the Kinesis Data Stream provides more capacity to handle the incoming data. Implementing retries with exponential backoff allows the Lambda function to retry failed operations without overwhelming the system, addressing transient issues effectively. These combined actions help resolve the throughput issue and ensure the pipeline resumes normal operation.
- A. Correct.
Increasing the shard count in the Kinesis Data Stream will provide more capacity for data ingestion and processing, reducing the likelihood of throughput errors. This is a valid solution.
- B. Incorrect.
While optimizing the Lambda function code might improve performance, it does not directly address the 'ProvisionedThroughputExceededException', which is related to Kinesis shard capacity.
- C. Incorrect.
S3 bucket versioning is unrelated to the issue of throughput in Kinesis and Lambda processing. It does not help resolve the problem.
- D. Correct.
Implementing retries with exponential backoff is a best practice for handling transient errors, such as 'ProvisionedThroughputExceededException'. This helps the Lambda function successfully process records after a brief wait.
- E. Incorrect.
Increasing the batch size in the Lambda function's event source mapping could worsen the problem by increasing the load on Kinesis shards, leading to more throughput issues.