Google Professional Machine Learning Engineer Question 395
Single answerGoogle Cloud PlatformYou are tasked with orchestrating a data preprocessing pipeline for a machine learning workflow in Google Cloud. The pipeline involves extracting data from Cloud Storage, transforming it using a Python script running on a Dataproc cluster, and loading the transformed data into BigQuery. You need a managed orchestration service that allows you to define and automate the workflow while providing monitoring and retry capabilities. Which Google Cloud service should you use?
- A
Cloud Composer
- B
Cloud Dataflow
- C
Cloud Functions
- D
BigQuery
Show answer and explanation
Correct answer: A
Explanation
The correct answer is Cloud Composer because it is specifically designed to orchestrate complex workflows across multiple Google Cloud services, such as Cloud Storage, Dataproc, and BigQuery. It provides monitoring, retry capabilities, and integration with other Google Cloud services, making it the ideal choice for the described machine learning pipeline.
- A. Correct.
Cloud Composer is a managed workflow orchestration service built on Apache Airflow. It is ideal for defining, scheduling, and monitoring workflows like the one described in the scenario.
- B. Incorrect.
Cloud Dataflow is a data processing service designed for stream and batch data transformations, but it is not designed for orchestrating multi-step workflows across services.
- C. Incorrect.
Cloud Functions is a serverless compute service that can execute single tasks but lacks the orchestration and workflow monitoring capabilities required for this scenario.
- D. Incorrect.
BigQuery is a data warehouse service for analytics but does not include workflow orchestration capabilities.