DEA-C01 Question 510
Single answerA company is building a real-time data processing pipeline using Amazon Kinesis Data Streams. The incoming data volume can vary significantly throughout the day, and the company wants to ensure cost-efficiency while maintaining the ability to handle peak loads. Which configuration approach should the data engineer choose when setting up the Kinesis Data Stream?
- A
Manually provision the number of shards based on the highest expected peak load
- B
Enable on-demand capacity mode for the Kinesis Data Stream
- C
Use Amazon Kinesis Data Firehose to automatically scale shards based on throughput
- D
Integrate AWS Auto Scaling with Kinesis Data Streams for dynamic shard management
Show answer and explanation
Correct answer: B
Explanation
On-demand capacity mode for Kinesis Data Streams is designed to automatically manage shard scaling based on the current traffic. This eliminates the need to manually provision or adjust shards and ensures both cost-efficiency and the ability to handle fluctuating data volumes. Other options either involve manual effort or misuse services not intended for this purpose.
- A. Incorrect.
Manually provisioning shards for the highest peak load ensures the stream can handle high traffic, but it is not cost-efficient during periods of low traffic because unused capacity will still incur costs.
- B. Correct.
Enabling on-demand capacity mode allows Kinesis Data Streams to automatically scale up or down based on traffic, ensuring cost-efficiency while handling varying loads without the need for manual intervention.
- C. Incorrect.
Amazon Kinesis Data Firehose is a delivery service and does not manage the scaling of shards in a Kinesis Data Stream; it is used for streaming data to destinations like Amazon S3 or Redshift.
- D. Incorrect.
Kinesis Data Streams does not natively support AWS Auto Scaling for shard management. Shard scaling must be managed manually or with the on-demand capacity mode.