DEA-C01 Question 34
Select 3You are designing a data ingestion pipeline on AWS to process streaming data from IoT devices. One of the key requirements is to ensure that the pipeline is replayable in case of downstream failure or data corruption. Which combination of approaches should you implement to enable replayability in your pipeline?
- A
Store raw events in Amazon S3 before processing them downstream.
- B
Use Amazon Kinesis Data Streams with the retention period set to the maximum allowed.
- C
Implement a distributed in-memory cache to temporarily store incoming data.
- D
Enable versioning on the Amazon S3 bucket used for raw data storage.
- E
Process data directly from Kinesis Data Streams without intermediate storage.
Show answer and explanation
Correct answers: A, B, D
Explanation
Ensuring replayability in a data ingestion pipeline involves implementing mechanisms to persistently store raw data and enabling the ability to retrieve and reprocess data when needed. Storing raw data in Amazon S3 and enabling versioning ensures that data is preserved, while extending retention in Kinesis Data Streams allows for replay within a specific time window. Temporary solutions like in-memory caches or bypassing persistent storage do not support long-term replayability.
- A. Correct.
Storing raw events in Amazon S3 ensures that the original data is preserved and can be reprocessed if needed, making the pipeline replayable.
- B. Correct.
Amazon Kinesis Data Streams with an extended retention period allows you to replay data within the retention window, supporting replayability.
- C. Incorrect.
An in-memory cache is volatile and not a reliable long-term storage solution for replayability needs.
- D. Correct.
Enabling versioning on the S3 bucket helps preserve different versions of the data, which can be useful for recovering or replaying older data.
- E. Incorrect.
Processing data directly from Kinesis without intermediate storage does not provide a persistent mechanism for replaying data if downstream systems fail.