DEA-C01 Question 451
Select 2You are designing a data pipeline for a real-time stock price analytics application. The pipeline needs to ingest, process, and store data with minimal latency. The architecture must ensure scalability and durability while allowing downstream consumers to access the processed data efficiently. Which of the following options would best meet this requirement?
- A
Use Amazon Kinesis Data Streams to ingest real-time stock price data, process it using AWS Lambda, and store the processed data in Amazon S3.
- B
Use Amazon SQS to ingest stock price data, process it using AWS Glue, and store the processed data in Amazon RDS.
- C
Use Amazon Kinesis Data Streams for ingestion and processing, and store the processed data in Amazon DynamoDB for low-latency querying.
- D
Use Amazon MSK (Managed Streaming for Apache Kafka) to ingest stock price data, process it with AWS Lambda, and store the processed data in Amazon Redshift.
- E
Use AWS IoT Core for ingestion, process the data with Amazon Kinesis Data Analytics, and store the processed data in Amazon ElastiCache.
Show answer and explanation
Correct answers: A, C
Explanation
The correct answers (1 and 3) leverage services optimized for real-time ingestion and processing, with scalable and durable storage solutions. Option 1 uses Amazon S3 for storage, which ensures durability and scalability but may not meet the lowest-latency querying requirements. Option 3, on the other hand, uses Amazon DynamoDB for storage, which provides low-latency querying capabilities, making it a better fit for the given scenario. Other options either misuse services or do not align with the real-time and low-latency requirements.
- A. Correct.
This option is correct because Amazon Kinesis Data Streams is designed for real-time data ingestion, AWS Lambda can process the data with low latency, and Amazon S3 provides scalable and durable storage for processed data. However, while this solution ensures durability, querying from S3 might not meet the lowest-latency requirement.
- B. Incorrect.
This option is incorrect because Amazon SQS is more suitable for message queuing rather than high-throughput real-time data ingestion, and AWS Glue is not designed for low-latency real-time processing.
- C. Correct.
This option is correct as it uses Amazon Kinesis Data Streams for real-time ingestion, which is highly scalable and low-latency. Storing processed data in Amazon DynamoDB provides low-latency querying, aligning with the requirements of the pipeline.
- D. Incorrect.
This option is incorrect because while Amazon MSK can handle real-time ingestion, storing processed data in Amazon Redshift is more suitable for analytical workloads rather than low-latency querying. This does not align with the requirements for minimal latency.
- E. Incorrect.
This option is incorrect because AWS IoT Core is primarily designed for IoT device data ingestion and not optimized for high-throughput real-time ingestion like stock price data. ElastiCache is also not a recommended long-term storage solution for processed data.