Google Professional Machine Learning Engineer Question 98
Select 2Google Cloud PlatformA company is building a recommendation system to predict customer preferences based on historical data. The dataset consists of petabytes of structured and semi-structured data stored in Google Cloud Storage. The team wants to preprocess the data, train a machine learning model, and ensure the solution is scalable for future growth. Which combination of technologies is most appropriate for this use case?
- A
Use Apache Spark on Dataproc for data preprocessing and training the model
- B
Use Cloud SQL to store the data and run SQL queries for preprocessing
- C
Use Spanner for storing and preprocessing the data before training the model
- D
Use Apache Hadoop on Dataproc to preprocess the data and export it for model training
- E
Use BigQuery ML to preprocess the data and train the model directly within BigQuery
Show answer and explanation
Correct answers: A, D
Explanation
For processing petabytes of structured and semi-structured data stored in Google Cloud Storage, distributed processing frameworks like Apache Spark and Apache Hadoop on Dataproc are ideal. They offer scalability and flexibility for data preprocessing and can be integrated with machine learning pipelines. Cloud SQL and Spanner are not suitable for this scale of data processing, and BigQuery ML is not designed for preprocessing data outside of BigQuery.
- A. Correct.
Correct: Apache Spark on Dataproc is highly suitable for processing and analyzing large-scale structured and semi-structured data. It is scalable and integrates well with Google Cloud Storage for preprocessing and distributed model training.
- B. Incorrect.
Incorrect: Cloud SQL is suitable for structured data but is not designed for handling petabytes of data or for performing large-scale data preprocessing and model training. It lacks the scalability required for this use case.
- C. Incorrect.
Incorrect: Spanner is optimized for globally-distributed, strongly-consistent transactional databases, not for preprocessing and machine learning model training on large-scale datasets.
- D. Correct.
Correct: Apache Hadoop on Dataproc is well-suited for processing massive datasets. It can be used to preprocess the data stored in Google Cloud Storage and export the results for model training.
- E. Incorrect.
Incorrect: BigQuery ML is a powerful tool for model training directly on data stored in BigQuery, but it is not designed for preprocessing petabytes of semi-structured data stored in Google Cloud Storage.