MLS-C01 Question 3
Single answerYou are building a machine learning pipeline to process large volumes of data stored in Amazon S3. The data is semi-structured and must be transformed, cleaned, and partitioned before being used for training a model. The transformed data must also be stored back in Amazon S3 for further processing. Which AWS service should you use to efficiently perform these data preparation tasks?
- A
AWS Glue
- B
Amazon Redshift
- C
Amazon EMR
- D
AWS Lambda
Show answer and explanation
Correct answer: A
Explanation
AWS Glue is the most appropriate service for this scenario because it is specifically designed for ETL workflows, supports semi-structured data, can partition data, and integrates seamlessly with Amazon S3. While Amazon EMR and AWS Lambda can also perform data transformations, they are less efficient or require more management for this use case.
- A. Correct.
AWS Glue is a managed ETL (Extract, Transform, Load) service that is designed to work with semi-structured data. It can clean, transform, and partition data, and it integrates seamlessly with Amazon S3, making it the most suitable choice for this scenario.
- B. Incorrect.
Amazon Redshift is a data warehouse service used for querying and analyzing large datasets, but it is not designed specifically for ETL tasks or working with raw semi-structured data stored in Amazon S3.
- C. Incorrect.
Amazon EMR is a big data processing service that can be used for complex transformations. However, it requires more setup and management compared to AWS Glue, which is serverless and optimized for ETL workflows.
- D. Incorrect.
AWS Lambda can be used for lightweight data transformations, but it has limitations in terms of execution time and is not ideal for processing large volumes of data or performing complex ETL tasks.