Google Professional Machine Learning Engineer Question 114
Select 2Google Cloud PlatformYour team is building a machine learning pipeline to process large amounts of raw customer transaction data stored in BigQuery. The pipeline needs to perform the following tasks: filter invalid records, normalize numerical features, and split the data into training and test sets. The solution must be scalable and automated. Which combination of tools would best address these requirements?
- A
Use Dataflow to filter invalid records and normalize numerical features, and split the data using a custom transformation.
- B
Use TensorFlow Extended (TFX) to design and deploy the pipeline, leveraging built-in components for data preprocessing and dataset splitting.
- C
Use BigQuery to filter invalid records and normalize numerical features, then export the data for further processing.
- D
Use a combination of Cloud Dataprep for preprocessing and manual scripting to split the data into training and test sets.
Show answer and explanation
Correct answers: A, B
Explanation
Dataflow and TFX are the most appropriate tools for this scenario. Dataflow excels at scalable data preprocessing, while TFX provides components specifically tailored for end-to-end machine learning workflows, including preprocessing and dataset splitting. Using BigQuery alone or relying on manual scripting with Cloud Dataprep would not meet the requirements for automation and scalability.
- A. Correct.
Dataflow is a managed service designed for scalable, serverless processing of data pipelines. It can handle filtering, normalization, and custom transformations efficiently, making it suitable for preprocessing tasks.
- B. Correct.
TFX is specifically designed for end-to-end ML workflows, including preprocessing tasks like filtering and normalization, as well as dataset splitting. Its components are optimized for automation and scalability.
- C. Incorrect.
BigQuery is a powerful tool for querying and analyzing data, but it is not ideal for building automated ML pipelines, especially for preprocessing tasks like normalization and dataset splitting.
- D. Incorrect.
While Cloud Dataprep can handle some preprocessing tasks, manual scripting introduces inefficiency and inconsistency, making it unsuitable for scalable and automated ML workflows.