DEA-C01 Question 74
Single answerA data engineering team is designing a pipeline to process massive amounts of sensor data from IoT devices. The data is ingested into AWS Kinesis Data Streams, processed in near real-time by AWS Lambda, and then written to Amazon Redshift for analytics. During a traffic surge, the team notices that Lambda is unable to keep up with the incoming data, causing data loss. They want to implement an intermediate staging solution to ensure no data is lost before processing. Which AWS service should they use as an intermediate data staging location?
- A
Amazon S3
- B
Amazon DynamoDB
- C
Amazon RDS
- D
Amazon ElastiCache
Show answer and explanation
Correct answer: A
Explanation
Amazon S3 is the best choice for intermediate data staging in a data pipeline. It provides high durability, scalability, and cost-effectiveness, making it ideal for temporarily storing unprocessed or partially processed data during traffic surges. Other options like DynamoDB, RDS, and ElastiCache are not designed for this use case and would not handle large-scale staging needs efficiently.
- A. Correct.
Amazon S3 is a highly durable and scalable object storage service that can serve as an intermediate staging location for storing large volumes of data. It integrates well with many AWS services, ensuring no data loss during traffic surges.
- B. Incorrect.
Amazon DynamoDB is a NoSQL database designed for high-speed, low-latency workloads, but it is not an ideal choice for intermediate data staging as it is not optimized for massive batch storage or streaming data.
- C. Incorrect.
Amazon RDS is a relational database service suited for transactional workloads, not for storing unstructured or semi-structured intermediate data in a data pipeline.
- D. Incorrect.
Amazon ElastiCache is an in-memory data store used for caching and low-latency access, which is not suitable for large-scale persistent data staging.