DEA-C01 Question 423
Select 2A company wants to process large volumes of semi-structured log data in near real-time and store the results in a data warehouse for analytical queries. They use Amazon Kinesis Data Streams to ingest the data. Which combination of services should the company use to process and write the data into the data warehouse efficiently?
- A
AWS Lambda and Amazon Redshift
- B
Amazon Kinesis Data Firehose and Amazon Redshift
- C
Amazon Kinesis Data Analytics and Amazon Redshift Spectrum
- D
Amazon Kinesis Data Firehose and Amazon S3
Show answer and explanation
Correct answers: B, D
Explanation
To ingest and process large volumes of semi-structured data in near real-time, Amazon Kinesis Data Firehose is the most appropriate service as it seamlessly integrates with Amazon Redshift and Amazon S3. Using Firehose with Redshift allows for direct ingestion into the data warehouse, while Firehose with S3 provides an alternative for querying data stored in S3 using tools like Redshift Spectrum. Together, these options align with the requirements for real-time processing and analytics.
- A. Incorrect.
AWS Lambda is suitable for lightweight event-driven processing, but it is not the most efficient solution for processing and writing large volumes of streaming data directly into Amazon Redshift.
- B. Correct.
Amazon Kinesis Data Firehose can directly ingest data from Amazon Kinesis Data Streams and write it to Amazon Redshift, making it an optimal choice for this scenario.
- C. Incorrect.
Amazon Kinesis Data Analytics is used for running SQL queries on streaming data, but Amazon Redshift Spectrum is designed for querying data in Amazon S3, not for storing data in Amazon Redshift.
- D. Correct.
Amazon Kinesis Data Firehose can load data into Amazon S3, which works well in a scenario where the data warehouse (e.g., Amazon Redshift) queries the data stored in S3. This provides additional flexibility for data storage and processing.