MLA-C01 Question 31
Select 3You are designing a real-time machine learning pipeline to process a high-throughput stream of clickstream data from a web application. The data needs to be ingested, pre-processed, and sent to an ML model hosted on Amazon SageMaker for inference. Which combination of AWS services would be most suitable for ingesting and processing this streaming data?
- A
Amazon Kinesis Data Streams for ingesting data and AWS Lambda for pre-processing
- B
Amazon Kinesis Data Firehose for ingesting data and pre-processing
- C
Apache Flink on Amazon Kinesis Data Analytics for pre-processing and Amazon Kinesis Data Streams for ingesting data
- D
Amazon Simple Queue Service (SQS) for ingesting data and Amazon SageMaker Processing for pre-processing
- E
Amazon Managed Streaming for Apache Kafka (MSK) for ingesting data and Apache Flink for pre-processing
Show answer and explanation
Correct answers: A, C, E
Explanation
To build a scalable, real-time machine learning pipeline for streaming data, you need services that can handle high-throughput data ingestion and provide real-time processing capabilities. Amazon Kinesis Data Streams, Amazon Managed Streaming for Apache Kafka (MSK), and Apache Flink on Amazon Kinesis Data Analytics are all designed for this purpose. AWS Lambda can also handle lightweight, event-driven pre-processing tasks. These services work together effectively to ensure a robust data pipeline capable of handling real-time streaming data.
- A. Correct.
Correct: Amazon Kinesis Data Streams is a managed service for ingesting high-throughput streaming data, and AWS Lambda can be used for lightweight event-driven pre-processing tasks.
- B. Incorrect.
Incorrect: Amazon Kinesis Data Firehose is primarily used for data delivery to storage destinations like S3 or Redshift and does not support advanced pre-processing directly.
- C. Correct.
Correct: Apache Flink on Amazon Kinesis Data Analytics provides powerful stream processing capabilities, and Amazon Kinesis Data Streams is a suitable service for ingesting high-throughput data streams.
- D. Incorrect.
Incorrect: Amazon SQS is a message queue service rather than a streaming service, and it is not designed for real-time high-throughput data streams. SageMaker Processing is used for batch data processing and not suitable for real-time stream pre-processing.
- E. Correct.
Correct: Amazon Managed Streaming for Apache Kafka (MSK) is a managed service for ingesting and managing Apache Kafka streams, and Apache Flink can be used for complex real-time stream processing.