MLS-C01 Question 43
Single answerYou are a Machine Learning Engineer working for a company that needs to preprocess a large dataset stored in Amazon S3 for a model training pipeline. The dataset contains a mix of JSON, CSV, and Parquet files. You need to create an ETL process to clean, normalize, and transform this data while ensuring scalability and cost-effectiveness. Which AWS Glue feature would best suit this requirement?
- A
AWS Glue Data Catalog
- B
AWS Glue DynamicFrame
- C
AWS Glue Crawlers
- D
AWS Glue Studio
Show answer and explanation
Correct answer: B
Explanation
AWS Glue DynamicFrame is the best choice here because it is designed for handling semi-structured and structured data, such as JSON, CSV, and Parquet files. It provides a distributed processing framework that is scalable and cost-effective for ETL operations. The other features, like the Data Catalog and Crawlers, are helpful for metadata management and discovery but do not directly address the need for data transformation.
- A. Incorrect.
AWS Glue Data Catalog is primarily used to store metadata information about datasets, but it does not directly help in preprocessing or transforming the data.
- B. Correct.
AWS Glue DynamicFrame is a suitable choice for this scenario as it is specifically designed to process semi-structured data like JSON, CSV, and Parquet. It provides scalability and simplifies the ETL process.
- C. Incorrect.
AWS Glue Crawlers are used to discover and catalog metadata for datasets but do not handle the actual ETL transformation of the data.
- D. Incorrect.
AWS Glue Studio is a visual tool to create and manage ETL jobs but does not directly process or transform the data itself. It provides a user-friendly interface but isn’t the core processing feature.