MLA-C01 Question 10
Select 2You are tasked with building a machine learning pipeline to process large volumes of structured and unstructured data from multiple sources, such as IoT devices and transactional databases. The data needs to be ingested in near real-time and stored in a scalable, fault-tolerant manner for downstream ML processing. Which combination of services should you use to meet these requirements?
- A
Amazon Kinesis Data Streams for real-time data ingestion and Amazon S3 for scalable storage
- B
AWS Lambda for real-time data ingestion and Amazon DynamoDB for storage
- C
Amazon Kinesis Firehose for real-time data transformation and Amazon S3 for storage
- D
Amazon Redshift for real-time ingestion and storage
- E
AWS Glue for real-time ingestion and Amazon S3 for storage
Show answer and explanation
Correct answers: A, C
Explanation
For this use case, you need real-time data ingestion and scalable storage to handle both structured and unstructured data from multiple sources. Amazon Kinesis Data Streams and Amazon Kinesis Firehose are both designed for real-time ingestion, while Amazon S3 provides the scalability and durability needed for storage. AWS Lambda, Amazon DynamoDB, Amazon Redshift, and AWS Glue do not fully meet the requirements of high-throughput, real-time ingestion or scalable storage for large volumes of unstructured data.
- A. Correct.
Amazon Kinesis Data Streams is designed for real-time ingestion of high-throughput data, and Amazon S3 provides scalable, durable storage that is ideal for storing large volumes of structured and unstructured data for ML workloads.
- B. Incorrect.
AWS Lambda is a serverless compute service and is not primarily used for high-throughput real-time data ingestion. While Amazon DynamoDB is a scalable NoSQL database, it is not the best choice for storing large volumes of unstructured data.
- C. Correct.
Amazon Kinesis Firehose is a managed service that can perform real-time data transformation and directly deliver the data to Amazon S3, making it an ideal choice for this use case.
- D. Incorrect.
Amazon Redshift is a data warehousing service that is optimized for analytics, not for real-time ingestion or storing unstructured data.
- E. Incorrect.
AWS Glue is primarily a data integration service for ETL workflows, not for real-time data ingestion. While Amazon S3 is suitable for storage, Glue does not meet the real-time ingestion requirement.