DEA-C01 Question 17
Single answerYou are a data engineer working for a retail company that needs to ingest sales transaction data into an Amazon S3 data lake. The data is generated by Point-of-Sale (POS) systems at multiple store locations. The business requires near real-time insights for inventory management, while retaining a complete history of all transactions for long-term analytics. Which ingestion pattern best meets these requirements?
- A
Use Amazon Kinesis Data Streams for real-time ingestion and store data in Amazon S3 as it arrives.
- B
Batch upload data from the POS systems to Amazon S3 every 24 hours.
- C
Use AWS Glue to schedule a daily ETL job to ingest data from the POS systems into Amazon S3.
- D
Use Amazon Kinesis Data Streams for real-time ingestion and periodically write the data into Amazon S3 partitions.
Show answer and explanation
Correct answer: D
Explanation
The correct answer is to use Amazon Kinesis Data Streams for real-time ingestion and periodically write the data into Amazon S3 partitions. This approach not only provides the near real-time insights needed for inventory management but also ensures that the historical data is stored in Amazon S3 for long-term analytics. The other options either fail to meet the near real-time requirement or neglect the need for periodic data storage in S3 for historical analysis.
- A. Incorrect.
This option provides near real-time ingestion through Amazon Kinesis Data Streams but does not mention periodic writes to Amazon S3, which is essential for long-term storage and analytics.
- B. Incorrect.
Batch uploading every 24 hours does not meet the requirement for near real-time insights, even though it retains data history.
- C. Incorrect.
This option schedules a daily ETL job, which does not fulfill the need for near real-time ingestion and insights.
- D. Correct.
This option combines near real-time ingestion using Amazon Kinesis Data Streams with periodic writes to Amazon S3, meeting both the requirements for near real-time insights and retaining a complete history for analytics.