MLA-C01 Question 33
Select 2You are building a real-time machine learning model that predicts stock price movements based on high-frequency trading data. The data is streamed in near real-time and needs to be processed continuously before being sent to an Amazon SageMaker endpoint for inference. Which combination of AWS services would you use to ingest and process the streaming data effectively?
- A
Amazon Kinesis Data Streams to ingest streaming data and AWS Lambda to process it
- B
Amazon Kinesis Data Firehose to ingest and transform data before storing it in Amazon S3
- C
Apache Flink on Amazon Kinesis Data Analytics to process streaming data in real-time
- D
Amazon Managed Streaming for Apache Kafka (Amazon MSK) to ingest streaming data and send it directly to the SageMaker endpoint
- E
Amazon Simple Queue Service (SQS) to queue streaming data and Amazon SageMaker for processing
Show answer and explanation
Correct answers: A, C
Explanation
For real-time machine learning workflows, using Amazon Kinesis Data Streams for low-latency data ingestion and AWS Lambda for processing, or Apache Flink on Amazon Kinesis Data Analytics for advanced stream processing, are the most appropriate solutions. These services are designed to handle high-frequency data streams and can integrate seamlessly with Amazon SageMaker for inference.
- A. Correct.
Amazon Kinesis Data Streams can ingest streaming data with low latency, and AWS Lambda can process this data in near real-time, making it a suitable option for real-time machine learning workflows.
- B. Incorrect.
Amazon Kinesis Data Firehose is used for ingesting and transforming streaming data before storing it in destinations like Amazon S3 or Redshift. While useful for batch processing or storage, it is not designed for real-time inference.
- C. Correct.
Apache Flink on Amazon Kinesis Data Analytics is purpose-built for real-time stream processing and allows you to perform sophisticated transformations and calculations on streaming data, making it ideal for a real-time machine learning workflow.
- D. Incorrect.
Amazon MSK is a managed service to run Apache Kafka, which can ingest streaming data. However, sending data directly from MSK to a SageMaker endpoint without intermediate processing is not a best practice for real-time stream processing.
- E. Incorrect.
Amazon SQS is a message queuing service designed for decoupling components, not for real-time stream processing. It is not suitable for ingesting or processing high-frequency streaming data.