MLS-C01 Question 63
Single answerYou are a data scientist working for an e-commerce company that processes terabytes of customer and transaction data daily. You need to prepare this data for machine learning by aggregating purchase histories and cleaning incomplete records. The data is stored in Amazon S3 in Parquet format. Which approach would be the most efficient for processing this ML-specific data at scale?
- A
Use Amazon EMR with Apache Spark to process and clean the data in parallel.
- B
Use an Amazon RDS instance to load, process, and clean the data using SQL queries.
- C
Download the data locally, process it using Python scripts, and upload it back to S3.
- D
Use AWS Glue with an ETL job to process the data and prepare it for ML.
Show answer and explanation
Correct answer: A
Explanation
Apache Spark on Amazon EMR is a highly efficient solution for processing large-scale datasets stored in Amazon S3, such as cleaning and aggregating data for ML purposes. It supports distributed processing and is well-suited for tasks like aggregations and transformations, making it the optimal choice in this scenario.
- A. Correct.
Correct. Apache Spark on Amazon EMR is optimized for distributed data processing at scale, making it efficient for handling large datasets and preparing them for ML.
- B. Incorrect.
Incorrect. Amazon RDS is not designed to handle large-scale data processing like Apache Spark. It is more suited for structured data and relational database tasks.
- C. Incorrect.
Incorrect. Downloading and processing terabytes of data locally is inefficient and would incur significant time and bandwidth costs.
- D. Incorrect.
Incorrect. AWS Glue is a great ETL tool, but it is not as efficient as Apache Spark on Amazon EMR for large-scale, complex ML-specific data transformations.