MLS-C01 Question 37
Single answerA company uses Amazon EMR to process large-scale log data for their machine learning workflows. The logs are stored in Amazon S3, and the company wants to optimize the performance of their EMR cluster while minimizing costs. They often perform iterative data processing and machine learning model training on this cluster. Which configuration is MOST suitable for this use case?
- A
Use spot instances with HDFS as the storage layer for the EMR cluster.
- B
Use on-demand instances with EMRFS to directly access data in Amazon S3.
- C
Use a combination of on-demand and spot instances with EMRFS and enable Amazon S3 consistent view.
- D
Use reserved instances with HDFS for faster local storage and iterative processing.
Show answer and explanation
Correct answer: C
Explanation
The use case requires a balance between performance and cost optimization for iterative processing and machine learning model training. A combination of on-demand and spot instances balances reliability and cost-effectiveness. EMRFS allows persistent storage in S3, and enabling Amazon S3 consistent view ensures data consistency during iterative operations. This configuration is best suited for the requirements of the scenario.
- A. Incorrect.
Using spot instances with HDFS could reduce costs, but it risks data loss since spot instances can be terminated at any time. HDFS is not ideal for iterative processing in this case as the data is stored locally and not persistent.
- B. Incorrect.
Using on-demand instances with EMRFS ensures persistent storage in Amazon S3, but it is not cost-effective. Additionally, without enabling Amazon S3 consistent view, there could be issues with eventual consistency during iterative processing.
- C. Correct.
Using a combination of on-demand and spot instances leverages cost savings from spot instances while maintaining reliability with on-demand instances. EMRFS ensures persistent storage in Amazon S3, and enabling Amazon S3 consistent view mitigates issues with eventual consistency, making this the optimal configuration for the use case.
- D. Incorrect.
Using reserved instances with HDFS provides cost savings and faster local storage, but HDFS is not suitable for iterative processing in this scenario because data stored locally on nodes is lost when nodes are scaled down or fail.