DEA-C01 Question 197
Select 2You are tasked with designing a real-time streaming data pipeline for a financial services company to process stock market data. The pipeline should ingest, process, and store streaming data with minimal latency. The company also wants to ensure the pipeline can handle unpredictable traffic spikes. Which combination of AWS services should you use to achieve this goal?
- A
Amazon Kinesis Data Streams for ingestion and AWS Lambda for processing.
- B
Amazon SQS for ingestion and Amazon RDS for storage.
- C
Amazon Kinesis Data Firehose for ingestion and Amazon Redshift for storage.
- D
Amazon Kinesis Data Streams for ingestion and Amazon DynamoDB for storage.
- E
Amazon Managed Streaming for Apache Kafka (Amazon MSK) for ingestion and Amazon S3 for storage.
Show answer and explanation
Correct answers: A, D
Explanation
To design a real-time streaming data pipeline, you need services that support low-latency ingestion, processing, and storage. Amazon Kinesis Data Streams is a managed service optimized for real-time streaming ingestion, and AWS Lambda can process the streaming data efficiently. For storage, Amazon DynamoDB is a highly scalable NoSQL database that can handle the demands of real-time data storage. While other options like Amazon SQS, Amazon RDS, and Amazon S3 are valuable AWS services, they are not tailored for real-time streaming scenarios.
- A. Correct.
Correct. Amazon Kinesis Data Streams is designed for real-time streaming data ingestion, and AWS Lambda can process the data with minimal latency. It is also scalable and can handle unpredictable traffic spikes.
- B. Incorrect.
Incorrect. Amazon SQS is a message queuing service, not designed for real-time streaming. Moreover, Amazon RDS is not optimal for storing high-throughput streaming data due to potential scalability limitations.
- C. Incorrect.
Incorrect. While Amazon Kinesis Data Firehose can ingest data, it is more suitable for near real-time delivery to storage destinations rather than real-time processing. Amazon Redshift is a data warehouse, which is not ideal for real-time storage of streaming data.
- D. Correct.
Correct. Amazon Kinesis Data Streams can handle real-time ingestion, and Amazon DynamoDB is highly scalable and low-latency, making it suitable for storing streaming data.
- E. Incorrect.
Incorrect. Amazon MSK can handle streaming data ingestion, but Amazon S3 is a bulk storage solution designed for large-scale data storage rather than real-time storage of streaming data.