MLS-C01 Question 17
Single answerA retail company needs to build a machine learning pipeline to predict customer purchase behavior. They have data arriving daily from multiple sources, including transactional databases, clickstream logs, and social media feeds. The data needs to be ingested into a central repository for preprocessing and feature engineering. Given the variety of data formats and the need to handle real-time and batch ingestion, which AWS solution is best suited to this requirement?
- A
Use AWS Glue to crawl the data from various sources and store it in Amazon S3.
- B
Use Amazon Kinesis Data Firehose for real-time streaming data ingestion and AWS Glue for batch data ingestion.
- C
Use AWS DataSync to transfer data from all sources to Amazon RDS for preprocessing.
- D
Use Amazon EMR to connect to the data sources and ingest the data directly into HDFS.
Show answer and explanation
Correct answer: B
Explanation
The combination of Amazon Kinesis Data Firehose and AWS Glue addresses both real-time and batch data ingestion requirements effectively. Amazon Kinesis Data Firehose can ingest real-time streaming data, while AWS Glue can handle batch data ingestion and data processing. This solution is also scalable and integrates well with other AWS services for machine learning workflows.
- A. Incorrect.
AWS Glue is suitable for batch data processing and cataloging but does not handle real-time ingestion. This option does not address the real-time data ingestion requirement.
- B. Correct.
Amazon Kinesis Data Firehose is designed for real-time streaming data ingestion, and AWS Glue can handle batch data ingestion. This combination provides a robust solution for both real-time and batch ingestion needs.
- C. Incorrect.
AWS DataSync is primarily used for transferring large datasets between on-premises storage and AWS. It is not designed for ingesting streaming data or handling diverse data formats.
- D. Incorrect.
Amazon EMR is mainly used for big data processing. While it can connect to various data sources, it is not optimized for real-time streaming ingestion and would require significant customization to meet the requirements.