MLA-C01 Question 335
Select 3You are tasked with designing a data ingestion pipeline for a machine learning application that processes real-time sensor data from IoT devices. The pipeline must automatically scale to handle bursts of data, orchestrate the flow of data between services, and ensure fault tolerance. Which combination of AWS services would best meet these requirements?
- A
AWS Kinesis Data Streams for real-time data ingestion and Amazon S3 for data storage
- B
AWS Step Functions for orchestration and error handling
- C
AWS Glue for real-time data ingestion and transformations
- D
Amazon EventBridge for monitoring and triggering workflows
- E
Amazon Managed Workflows for Apache Airflow (MWAA) for orchestration
Show answer and explanation
Correct answers: A, B, E
Explanation
For a real-time data ingestion pipeline, AWS Kinesis Data Streams ensures the system can handle bursts of IoT sensor data, and Amazon S3 provides scalable storage. AWS Step Functions and MWAA are excellent options for orchestrating workflows, ensuring fault tolerance, and managing the flow of data between services. AWS Glue, while powerful, is more suited for batch processing, and EventBridge is primarily for event-driven architectures rather than orchestration of complex pipelines.
- A. Correct.
AWS Kinesis Data Streams is designed for real-time data ingestion, making it suitable for handling bursts of sensor data. Amazon S3 provides durable, scalable storage for the ingested data.
- B. Correct.
AWS Step Functions is a serverless orchestration service capable of managing workflows and handling errors, crucial for ensuring fault tolerance.
- C. Incorrect.
AWS Glue is a managed ETL service, but it is not optimized for real-time data ingestion. It is better suited for batch processing rather than real-time scenarios.
- D. Incorrect.
Amazon EventBridge is primarily used for event-driven applications and monitoring. While it can trigger workflows, it is not designed to orchestrate complex data pipelines.
- E. Correct.
Amazon Managed Workflows for Apache Airflow (MWAA) is a managed orchestration service that supports complex workflows, making it a good candidate for coordinating data flow in a machine learning pipeline.