MLS-C01 Question 26
Select 3You are designing a data ingestion pipeline for a machine learning model that predicts customer churn. The pipeline must handle both batch data from monthly reports and real-time streaming data from customer interactions on the platform. Which combination of AWS services would best allow you to orchestrate and manage this pipeline?
- A
Amazon Kinesis Data Streams for streaming data ingestion and Amazon Glue for batch data processing
- B
AWS Step Functions to orchestrate the batch and streaming workflows
- C
Amazon S3 for storing both batch and streaming data and Amazon QuickSight for visualization
- D
Amazon EMR for batch processing and AWS Lambda for processing streaming data
- E
Amazon Redshift for real-time streaming data ingestion and batch data storage
Show answer and explanation
Correct answers: A, B, D
Explanation
To orchestrate a data ingestion pipeline that handles both batch and streaming workloads, you need services optimized for these respective tasks. Amazon Kinesis Data Streams and AWS Lambda are great for real-time streaming ingestion and processing, while Amazon Glue and Amazon EMR are strong candidates for batch processing. AWS Step Functions can serve as the orchestrator to manage and coordinate these workflows effectively. Amazon S3 is commonly used for storage but does not orchestrate workflows, and Amazon QuickSight and Amazon Redshift are not directly relevant for this use case in terms of ingestion and orchestration.
- A. Correct.
Correct: Amazon Kinesis Data Streams is designed for real-time streaming data ingestion, and Amazon Glue is ideal for ETL tasks in batch processing workflows.
- B. Correct.
Correct: AWS Step Functions is a serverless orchestration service that can coordinate both batch and streaming workflows in a single pipeline.
- C. Incorrect.
Incorrect: While Amazon S3 is a good choice for storage, Amazon QuickSight is a visualization tool and does not contribute to orchestrating the pipeline or managing ingestion effectively.
- D. Correct.
Correct: Amazon EMR is well-suited for large-scale batch data processing, and AWS Lambda can process streaming data events in real time.
- E. Incorrect.
Incorrect: Amazon Redshift is a data warehouse solution optimized for analytics rather than real-time streaming data ingestion or orchestration.