Google Professional Machine Learning Engineer Question 240
Select 2Google Cloud PlatformYou are designing a data pipeline to ingest various types of files (CSV, JSON, images) into Google Cloud for training a machine learning model. The pipeline must handle structured and unstructured data, ensure scalability, and integrate seamlessly with BigQuery for analytics. Which combination of Google Cloud services would be the most appropriate for this task?
- A
Cloud Storage for storing files and Cloud Dataflow for processing
- B
BigQuery ML for directly ingesting the files
- C
Dataproc for batch processing and Cloud Storage for storing intermediate data
- D
Pub/Sub for streaming ingestion and Cloud Dataflow for processing
- E
Cloud SQL for ingesting image and JSON files
Show answer and explanation
Correct answers: A, D
Explanation
The combination of Cloud Storage and Cloud Dataflow is optimal for handling diverse file types like CSV, JSON, and images. Cloud Storage provides scalable and durable file storage, while Cloud Dataflow can process this data efficiently in both batch and streaming modes. Additionally, using Pub/Sub with Cloud Dataflow allows for seamless streaming data ingestion, making it a flexible solution for real-time and batch processing. BigQuery can subsequently be used for advanced analytics once the data is processed and ingested.
- A. Correct.
Cloud Storage is ideal for storing various file types, and Cloud Dataflow is a fully managed stream and batch data processing service that can handle structured and unstructured data. This is a scalable and efficient solution.
- B. Incorrect.
BigQuery ML is used for building and deploying machine learning models directly in BigQuery but is not designed for ingesting diverse file types like images or JSON files.
- C. Incorrect.
Dataproc is suitable for processing large-scale data using Hadoop or Spark but is less optimal for handling streaming data or integrating with BigQuery directly.
- D. Correct.
Pub/Sub is a messaging service that supports streaming data ingestion, and integrating it with Cloud Dataflow allows for real-time or batch processing of structured and unstructured data. This is a scalable and flexible option for the pipeline.
- E. Incorrect.
Cloud SQL is a relational database service and is not designed for ingesting unstructured data like images or JSON files. It is not suitable for this use case.