MLA-C01 Question 37
Single answerYou are designing a machine learning pipeline to train and deploy a model for predicting customer churn. The dataset is 2 TB in size and needs to be preprocessed before training. The preprocessing step will involve distributed processing using AWS Glue, and the model training will be performed using Amazon SageMaker. The processed data will be accessed frequently during training and needs low latency. Which AWS storage option should you use to store the processed data?
- A
Amazon S3
- B
Amazon EFS
- C
Amazon FSx for Lustre
- D
Amazon DynamoDB
Show answer and explanation
Correct answer: C
Explanation
Amazon FSx for Lustre is the best choice for storing processed data in this scenario because it is specifically optimized for high-performance computing workloads, such as machine learning training. It integrates seamlessly with Amazon S3, allowing you to store raw data in S3 and automatically load it into FSx for Lustre for fast, low-latency access during training. Other options, such as Amazon S3 and Amazon EFS, provide scalable storage but lack the performance characteristics required for this use case.
- A. Incorrect.
Amazon S3 is a cost-effective storage service suitable for large-scale data storage. However, it is not optimized for low-latency, high-performance access required during training, especially for distributed workloads.
- B. Incorrect.
Amazon EFS provides scalable, shared file storage but is not optimized for high-performance workloads like machine learning training. Its performance characteristics may not meet the low-latency requirements during training.
- C. Correct.
Amazon FSx for Lustre is designed for high-performance workloads such as machine learning training. It provides low-latency, high-throughput storage, making it ideal for accessing frequently used datasets during training.
- D. Incorrect.
Amazon DynamoDB is a NoSQL database service optimized for key-value or document-based workloads. It is not suitable for storing large datasets or high-throughput access patterns needed for machine learning training.