Google Professional Machine Learning Engineer Question 174
Select 3Google Cloud PlatformYou are designing a distributed machine learning pipeline for a large-scale recommendation system on Google Cloud. The pipeline involves data preprocessing, training, and batch inference. The dataset is too large to fit into the memory of a single machine, and the model training requires GPU acceleration. Which frameworks should you use for this scenario?
- A
TensorFlow for model training and Spark for data preprocessing
- B
PyTorch for model training and sklearn for data preprocessing
- C
TensorFlow for model training and JAX for data preprocessing
- D
TensorFlow for model training and Spark for both data preprocessing and batch inference
- E
PyTorch for model training and Spark for data preprocessing
Show answer and explanation
Correct answers: A, D, E
Explanation
When designing a distributed machine learning pipeline on Google Cloud, the frameworks must align with the requirements of scalability and performance. TensorFlow and PyTorch are both capable of utilizing GPUs for model training, while Spark is ideal for distributed data preprocessing and batch inference on large datasets. Sklearn and JAX, while powerful in specific use cases, lack the scalability required for this scenario.
- A. Correct.
TensorFlow is well-suited for GPU-accelerated model training, and Spark is optimal for distributed data preprocessing when dealing with very large datasets.
- B. Incorrect.
PyTorch is a strong choice for GPU-accelerated model training, but sklearn is not designed for distributed data preprocessing, making this combination unsuitable for the given scenario.
- C. Incorrect.
TensorFlow is excellent for GPU training, but JAX is not designed for distributed data preprocessing, which makes this combination inappropriate for handling large-scale datasets.
- D. Correct.
This is a valid option because TensorFlow can handle GPU-accelerated training, and Spark is capable of distributed data preprocessing and also supports batch inference at scale.
- E. Correct.
PyTorch can perform GPU-accelerated model training, while Spark is effective for distributed data preprocessing, making this a suitable choice for the scenario.