MLS-C01 Question 38
Single answerYou are a data scientist working for an e-commerce company. You need to process and analyze terabytes of clickstream data stored in Amazon S3 to generate user behavior insights. The analysis requires building a Spark-based machine learning pipeline. Security is a key concern, and the data must not leave the AWS environment. Which approach would be the most suitable for this task?
- A
Use Amazon EMR to create a managed Spark cluster, process the data in Amazon S3 using EMRFS, and configure encryption at rest and in transit.
- B
Use an on-premises Apache Spark cluster to process the data after downloading it from Amazon S3.
- C
Use AWS Glue to process the data in Amazon S3 and then move the processed data to Amazon RDS for analysis.
- D
Use Amazon SageMaker to directly process the raw clickstream data stored in Amazon S3 without using a cluster.
Show answer and explanation
Correct answer: A
Explanation
Amazon EMR is a managed service that simplifies running big data frameworks like Apache Spark on AWS infrastructure. It integrates seamlessly with Amazon S3 for data storage and allows for security configurations such as encryption at rest and in transit, meeting the requirements of this scenario. The other options either do not meet the security requirements or are not optimal for processing large-scale clickstream data.
- A. Correct.
This is the correct answer. Amazon EMR is specifically designed for big data processing and supports Apache Spark. EMRFS can directly access data in Amazon S3, and you can configure encryption for data security.
- B. Incorrect.
This is incorrect because using an on-premises cluster requires data transfer out of AWS, which violates the security requirement.
- C. Incorrect.
This is incorrect because AWS Glue is more suited for ETL tasks, and moving data to Amazon RDS is not optimal for large-scale machine learning pipelines.
- D. Incorrect.
This is incorrect because Amazon SageMaker is better suited for building and training ML models, not for raw big data processing at scale.