Google Professional Machine Learning Engineer Question 115
Single answerGoogle Cloud PlatformYou are tasked with building a machine learning pipeline on Google Cloud to preprocess a dataset containing millions of records stored in BigQuery. The dataset requires cleaning, normalization, and feature engineering before being used to train a TensorFlow model. Which approach should you use to preprocess the data efficiently and scalably?
- A
Export the data from BigQuery to CSV files, then preprocess it using pandas and NumPy before saving the results to Cloud Storage.
- B
Use TensorFlow Extended (TFX) with a Dataflow pipeline to preprocess the data directly from BigQuery.
- C
Write a custom Python script to preprocess the data and run it on a Compute Engine instance.
- D
Use BigQuery's built-in SQL capabilities to preprocess the data and export the results to TensorFlow.
Show answer and explanation
Correct answer: B
Explanation
Using TensorFlow Extended (TFX) with Dataflow is the best approach for preprocessing large-scale datasets in Google Cloud. It provides scalability, automation, and seamless integration with BigQuery and TensorFlow, making it ideal for building end-to-end machine learning pipelines. Other methods either lack scalability, require significant manual effort, or do not integrate well with Google Cloud services.
- A. Incorrect.
Exporting data to CSV and using pandas/NumPy is not scalable for datasets containing millions of records. This method would require substantial manual effort and could lead to inefficiencies in handling large-scale data.
- B. Correct.
TFX combined with Dataflow is the recommended approach for scalable and efficient preprocessing of large datasets. This solution integrates seamlessly with BigQuery, processes data in parallel, and prepares it for TensorFlow models in an automated way.
- C. Incorrect.
A custom Python script on Compute Engine can preprocess the data, but it does not provide built-in scalability or efficiency for handling large-scale datasets. Additionally, this approach lacks integration with other Google Cloud services like BigQuery and TensorFlow.
- D. Incorrect.
While BigQuery's SQL capabilities can preprocess data, this approach is limited in terms of advanced feature engineering and automation. It also does not integrate seamlessly with TensorFlow for a machine learning pipeline.