DEA-C01 Question 307
Select 2You are a data engineer tasked with processing large-scale event data from IoT devices in real-time. You use Amazon Kinesis Data Streams to ingest the data and AWS Lambda for processing. During testing, you notice that Lambda functions are being throttled due to a high volume of API calls to Kinesis. What steps should you take to resolve this issue while ensuring efficient data processing?
- A
Increase the shard count in your Kinesis Data Stream to handle higher throughput.
- B
Enable Lambda Reserved Concurrency to limit the number of concurrent Lambda invocations.
- C
Use Kinesis Data Firehose instead of Kinesis Data Streams for real-time processing.
- D
Implement batching in your Lambda function to process multiple records in a single API call.
- E
Use DynamoDB Streams to replace Kinesis Data Streams for event ingestion.
Show answer and explanation
Correct answers: A, D
Explanation
To resolve the throttling issue, increasing the shard count in Kinesis Data Streams ensures higher throughput and capacity for API calls. Additionally, implementing batching in the Lambda function reduces the number of API calls needed to process the data, improving efficiency. These steps directly address the root cause of the throttling while supporting real-time data processing requirements.
- A. Correct.
Increasing the shard count in Kinesis Data Streams will allow more capacity for API calls and handle higher throughput, helping to reduce throttling.
- B. Incorrect.
Enabling Lambda Reserved Concurrency limits the number of concurrent Lambda executions, but it does not address the root cause of throttling due to high API call volumes.
- C. Incorrect.
Kinesis Data Firehose is mainly used for near real-time data delivery to destinations such as S3 and Redshift, not for real-time processing with Lambda.
- D. Correct.
Batching allows multiple records to be processed in one API call, reducing the frequency of calls to Kinesis Data Streams and minimizing throttling.
- E. Incorrect.
DynamoDB Streams is not a suitable replacement for Kinesis Data Streams in this scenario, as it is designed for change data capture in DynamoDB tables, not for high-throughput IoT event ingestion.