Google Professional Machine Learning Engineer Question 118
Single answerGoogle Cloud PlatformYou are tasked with building a machine learning pipeline for a recommendation system. The dataset resides in BigQuery and contains millions of records. You need to preprocess the data to handle missing values, normalize numerical features, and split the dataset into training, validation, and test sets. Which of the following approach is the most appropriate for this task?
- A
Use TensorFlow Extended (TFX) to create a preprocessing pipeline that reads data from BigQuery, applies transformations, and outputs the splits.
- B
Export the BigQuery data to CSV files and use a Python script with Pandas to preprocess the data, then split it into training, validation, and test sets.
- C
Directly query the data from BigQuery into a TensorFlow Dataset, preprocess it using TensorFlow functions, and split it into different datasets.
- D
Use Dataflow to preprocess the data, handle missing values, normalize features, and split the dataset into training, validation, and test sets.
Show answer and explanation
Correct answer: A
Explanation
TFX is the most appropriate choice for this scenario because it is designed to handle end-to-end ML pipelines, including preprocessing, with seamless integration with BigQuery. It supports large-scale data processing and provides tools for splitting datasets, ensuring a streamlined and scalable workflow. Other approaches either lack scalability, efficiency, or the necessary ML-centric features.
- A. Correct.
Correct. TensorFlow Extended (TFX) is specifically designed to build end-to-end ML pipelines, including preprocessing tasks and handling large-scale data from sources like BigQuery efficiently. It is the most scalable and maintainable choice for this task.
- B. Incorrect.
Not ideal. Exporting BigQuery data to CSV and using Pandas is not scalable for large datasets, as Pandas operates in-memory. This approach would likely cause performance issues and is not suitable for production-grade pipelines.
- C. Incorrect.
Not correct. While TensorFlow Dataset can be used for preprocessing, it is not optimal for handling large-scale data directly from BigQuery. Additionally, splitting the dataset into training, validation, and test sets is not inherently supported in this approach.
- D. Incorrect.
Partially correct but not the best choice. Dataflow is capable of preprocessing large datasets, but it lacks the specific integrations and ML-focused features of TFX. Using TFX provides a more comprehensive and maintainable solution for building ML pipelines.