MLS-C01 Question 64
Single answerYou are a data scientist working on a machine learning pipeline for a large retail company. Your dataset is stored on Amazon S3 and consists of millions of records. You need to preprocess the data by filtering out incomplete rows, aggregating sales data by regions, and normalizing numerical features. Given the scale of the data, which framework should you use to efficiently handle this preprocessing task?
- A
AWS Glue with Apache Spark
- B
Amazon SageMaker Processing with a Python script
- C
Amazon RDS with SQL queries
- D
Amazon QuickSight for data analysis
Show answer and explanation
Correct answer: A
Explanation
AWS Glue with Apache Spark is the most appropriate choice for preprocessing a large-scale dataset stored in Amazon S3. It provides a scalable and distributed computing environment for tasks such as filtering, aggregation, and normalization. Apache Spark’s MapReduce capabilities make it highly efficient for handling big data workflows.
- A. Correct.
AWS Glue with Apache Spark is well-suited for handling large-scale data preprocessing tasks. It can efficiently handle MapReduce operations such as filtering, aggregating, and normalizing data using Spark's distributed computing capabilities.
- B. Incorrect.
Amazon SageMaker Processing is an option for data preprocessing but is not as efficient for handling very large datasets compared to a distributed framework like Apache Spark.
- C. Incorrect.
Amazon RDS is a relational database service that supports SQL queries. While it can process data, it is not designed for distributed, large-scale data preprocessing tasks like those involving MapReduce.
- D. Incorrect.
Amazon QuickSight is a business intelligence tool for data visualization and analysis, not for preprocessing large-scale datasets.