DEA-C01 Question 299
Select 2A company is using Amazon Kinesis Data Streams to collect log data from its application in real-time. The data engineering team wants to process this stream using AWS services to analyze the logs, apply transformations, and store the processed data in Amazon S3 for long-term storage. Which combination of services could achieve this with minimal operational overhead?
- A
Use AWS Lambda to process the data in real-time and write the output to Amazon S3
- B
Use Amazon Kinesis Data Firehose to directly ingest the stream and deliver it to Amazon S3
- C
Use Amazon EMR to process the data and save the results to Amazon S3
- D
Use AWS Glue streaming jobs to transform the data and store it in Amazon S3
- E
Use Amazon Redshift to process the data from Kinesis Data Streams and export it to Amazon S3
Show answer and explanation
Correct answers: B, D
Explanation
The combination of Amazon Kinesis Data Firehose and AWS Glue streaming jobs provides a low operational overhead solution for processing and transforming streaming data from Kinesis Data Streams and storing it in Amazon S3. Kinesis Data Firehose offers a direct integration with S3, while AWS Glue streaming jobs handle complex transformations efficiently. Other options like AWS Lambda, Amazon EMR, and Amazon Redshift either require additional setup or are not designed for this specific use case.
- A. Incorrect.
AWS Lambda can process data from Kinesis Data Streams, but it requires setting up explicit logic for writing the output to S3. While possible, it may require more operational overhead compared to other services.
- B. Correct.
Amazon Kinesis Data Firehose is designed to directly ingest streaming data and deliver it to destinations like Amazon S3 without requiring custom code, making it an ideal choice for this use case.
- C. Incorrect.
Amazon EMR can process data from Kinesis Data Streams, but it involves managing clusters, which increases operational overhead. This is not the most efficient option for this scenario.
- D. Correct.
AWS Glue streaming jobs are designed for processing and transforming streaming data, and they can write the transformed data directly to Amazon S3, making it a suitable option with minimal operational overhead.
- E. Incorrect.
Amazon Redshift is a data warehouse service and is not designed for directly processing streaming data from Kinesis Data Streams. This would require additional steps and is not the best fit for the scenario.