MLA-C01 Question 29
Single answerYou are tasked with building a machine learning model to analyze large image datasets. The dataset is stored on Amazon S3. As part of your training pipeline, you need to preprocess the images in parallel using a distributed machine learning framework like TensorFlow. The preprocessing step requires high-throughput, low-latency access to the image data. Which storage solution should you use to maximize performance during preprocessing?
- A
Continue using Amazon S3 to directly load the images into your training instances
- B
Use Amazon EFS to store the images and mount it to your training instances
- C
Use Amazon FSx for NetApp ONTAP and mount it to your training instances
- D
Copy the images from Amazon S3 to the instance store volume of the training instances
Show answer and explanation
Correct answer: C
Explanation
Amazon FSx for NetApp ONTAP is the optimal choice for this scenario because it provides the necessary high-throughput and low-latency access to the dataset while minimizing operational overhead. It is better suited for distributed machine learning preprocessing tasks compared to Amazon S3, Amazon EFS, or instance store volumes. This makes it a reliable and performant solution for high-scale preprocessing workflows.
- A. Incorrect.
While Amazon S3 is highly scalable and cost-effective for storage, it is not optimized for low-latency, high-throughput access necessary for distributed preprocessing workflows. Accessing S3 directly may lead to throttling and increased latency.
- B. Incorrect.
Amazon EFS provides a scalable file storage solution for Linux-based applications, but it is better suited for general-purpose file sharing and does not provide the same level of performance and advanced features as FSx for NetApp ONTAP for machine learning workloads.
- C. Correct.
Amazon FSx for NetApp ONTAP offers high-performance file storage with support for low-latency, high-throughput workloads. It is well-suited for distributed machine learning frameworks that need consistent access to large datasets like images.
- D. Incorrect.
Using instance store volumes may offer high throughput and low latency, but it requires manually copying data from Amazon S3 for each training job, which is operationally inefficient. Additionally, instance store data is ephemeral and will be lost if the instance is stopped.