Google Professional Machine Learning Engineer Question 397
Single answerGoogle Cloud PlatformYou are a Machine Learning Engineer tasked with orchestrating a machine learning pipeline using Google Cloud Composer. The pipeline includes data preprocessing, model training, and model evaluation. The model training step requires access to a large dataset stored in BigQuery and should execute in a custom container. How can you configure this step in the pipeline using Composer?
- A
Use a BigQueryOperator to query the data and a KubernetesPodOperator to execute the model training in a custom container.
- B
Use a DataflowOperator to read the data from BigQuery and execute the model training directly within Dataflow.
- C
Use a PythonOperator to execute a Python script that reads data from BigQuery and trains the model locally within the Composer environment.
- D
Use a BigQueryOperator to export the data to Cloud Storage, then use a KubernetesPodOperator to train the model in the custom container using the exported data.
Show answer and explanation
Correct answer: A
Explanation
The BigQueryOperator efficiently handles querying the dataset from BigQuery, while the KubernetesPodOperator allows you to run custom containerized tasks such as model training. This configuration ensures scalability, modularity, and proper utilization of Google Cloud services. The other options either misuse the tools or introduce unnecessary complexity.
- A. Correct.
This is the correct approach because the BigQueryOperator can handle querying the data from BigQuery, and the KubernetesPodOperator is designed to run tasks in custom containers, making it ideal for model training in this case.
- B. Incorrect.
This is incorrect because DataflowOperator is primarily used for orchestrating Apache Beam pipelines and is not suitable for executing custom container-based tasks like model training.
- C. Incorrect.
This is incorrect because running a Python script locally within the Composer environment is not scalable and does not leverage the custom container required for model training.
- D. Incorrect.
This is incorrect because exporting data to Cloud Storage using BigQueryOperator adds unnecessary overhead when the data can be accessed directly from BigQuery.