DEA-C01 Question 386
Single answerYou are designing a data pipeline for a retail company that collects real-time sales transactions from multiple stores across the globe. The data is ingested into an Amazon Kinesis Data Stream and needs to be processed in near real-time before being stored in Amazon S3 for analytics. Which service should you use to process and transform the data in real-time?
- A
AWS Glue
- B
Amazon Kinesis Data Analytics
- C
Amazon EMR
- D
Amazon Redshift
Show answer and explanation
Correct answer: B
Explanation
Amazon Kinesis Data Analytics is the best service for processing and transforming streaming data in real-time. It integrates seamlessly with Amazon Kinesis Data Streams and offers SQL-based and Apache Flink-based processing, making it ideal for near real-time analytics and transformations before the data is stored in Amazon S3. Other options like Glue, EMR, and Redshift are either designed for batch processing or analytics on stored data, and are not suitable for real-time processing of streaming data.
- A. Incorrect.
AWS Glue is primarily used for ETL (Extract, Transform, Load) processes on batch data and is not ideal for real-time streaming data processing.
- B. Correct.
Amazon Kinesis Data Analytics is designed to process and analyze streaming data in real-time using SQL or Apache Flink, making it the best choice for this scenario.
- C. Incorrect.
Amazon EMR is mainly used for big data processing, including batch data processing, and is not optimized for real-time data transformations.
- D. Incorrect.
Amazon Redshift is a data warehouse service designed for querying and analyzing structured data but is not used for real-time data processing.