MLS-C01 Question 1
Single answerYou are working as a data engineer at an e-commerce company and are tasked with designing a pipeline to preprocess large volumes of customer transaction data stored in Amazon S3. The preprocessing involves filtering, aggregating, and transforming the data before it is used for training a machine learning model. The process should be scalable, cost-efficient, and able to handle growing data volumes. Which of the following AWS services would be most suitable for this task?
- A
AWS Glue
- B
Amazon EMR
- C
AWS Lambda
- D
Amazon Redshift
Show answer and explanation
Correct answer: A
Explanation
AWS Glue is the most suitable service for this scenario because it is specifically designed for scalable and cost-efficient ETL tasks, such as filtering, aggregating, and transforming large volumes of data stored in Amazon S3. It simplifies the management of data preprocessing pipelines and integrates seamlessly with other AWS services, making it an ideal choice for preparing data for machine learning workflows.
- A. Correct.
AWS Glue is a fully managed ETL (Extract, Transform, Load) service that is ideal for preprocessing large volumes of data stored in Amazon S3. It is scalable, cost-efficient, and integrates well with AWS services for machine learning workflows.
- B. Incorrect.
Amazon EMR is a managed Hadoop framework that can also preprocess data, but it requires more configuration and management effort compared to AWS Glue. While it is scalable, it is generally used for more complex big data processing tasks.
- C. Incorrect.
AWS Lambda is a serverless compute service that could preprocess data, but it has limitations in terms of execution time (15 minutes per invocation) and memory, making it unsuitable for large-scale data preprocessing.
- D. Incorrect.
Amazon Redshift is a data warehousing service optimized for analytical queries, not for preprocessing unstructured or semi-structured data in an ETL pipeline.