DEA-C01 Question 4
Single answerYou are building a data ingestion pipeline for a financial analytics application. The pipeline needs to ingest real-time stock market data from an external API into Amazon S3 for further processing. The data needs to be processed with minimal latency. Which solution should you implement to achieve this?
- A
Use Amazon Kinesis Data Firehose to directly ingest and deliver the data into an S3 bucket.
- B
Use AWS Glue to extract the data from the external API and save it to Amazon S3.
- C
Use Amazon Simple Queue Service (SQS) to buffer the data and then write a Lambda function to process and store it in S3.
- D
Use an EC2 instance to periodically call the API, process the data locally, and upload it to Amazon S3.
Show answer and explanation
Correct answer: A
Explanation
Amazon Kinesis Data Firehose is the best solution for ingesting real-time data into Amazon S3 with minimal latency. It is a fully managed service designed for real-time data streaming and delivery, making it ideal for use cases like real-time stock market data ingestion. Other options like AWS Glue and SQS are more suited for batch processing or message queuing, and using an EC2 instance adds unnecessary complexity and latency.
- A. Correct.
Correct. Amazon Kinesis Data Firehose is designed for real-time data ingestion and can directly deliver data to S3 with low latency.
- B. Incorrect.
Incorrect. AWS Glue is primarily used for ETL (Extract, Transform, Load) operations and not ideal for real-time ingestion from an external API.
- C. Incorrect.
Incorrect. While SQS can be used for buffering, it introduces additional overhead and latency, making it less suitable for real-time ingestion.
- D. Incorrect.
Incorrect. Using an EC2 instance for real-time ingestion is less efficient and introduces higher latency compared to managed services like Kinesis Data Firehose.