Google Professional Machine Learning Engineer Question 390
Single answerGoogle Cloud PlatformYou are designing a machine learning workflow to train and deploy a model using Google Cloud. Your team requires the following capabilities: managing pipeline steps, tracking metadata, and scaling workloads dynamically on Kubernetes. Which orchestration framework should you choose to best meet these requirements?
- A
Vertex AI Pipelines
- B
Kubeflow Pipelines
- C
Apache Airflow
- D
Cloud Composer
Show answer and explanation
Correct answer: B
Explanation
Kubeflow Pipelines is an open-source orchestration framework that is designed to run on Kubernetes. It provides features like metadata tracking, pipeline management, and dynamic scaling of workloads, making it well-suited for machine learning workflows that need to leverage Kubernetes. While Vertex AI Pipelines is also a strong contender, it does not provide the same level of direct Kubernetes integration or dynamic scaling as Kubeflow Pipelines, which is critical in this scenario.
- A. Incorrect.
Vertex AI Pipelines is primarily designed for managing machine learning workflows on Google Cloud but does not natively rely on Kubernetes for dynamic scaling. It is better suited for users who prefer a managed service with less complexity in managing Kubernetes clusters.
- B. Correct.
Kubeflow Pipelines is the best choice for this scenario as it is specifically built for orchestrating ML workflows on Kubernetes. It provides features like metadata tracking and dynamic scaling of workloads using Kubernetes, which aligns with the requirements.
- C. Incorrect.
Apache Airflow is a general-purpose workflow orchestration tool that can manage various workflows, but it is not specifically designed for ML pipelines and lacks native support for metadata tracking and Kubernetes-based dynamic scaling tailored for ML workflows.
- D. Incorrect.
Cloud Composer is a managed orchestration service based on Apache Airflow, which similarly lacks features specific to ML workflows, such as metadata tracking or Kubernetes-based dynamic scaling.