Google Professional Data Engineer Question 65
Select 1Google Cloud PlatformYour team is building a real-time analytics pipeline on Google Cloud that ingests streaming data from IoT devices and processes it for anomaly detection. The data needs to be ingested with low latency and processed in near real-time. After processing, the results should be stored for further analysis using BigQuery. Which combination of tools should you use to meet these requirements?
- A
Cloud Pub/Sub for data ingestion, Dataflow for processing, and BigQuery for storage
- B
Cloud Storage for data ingestion, Dataproc for processing, and BigQuery for storage
- C
Cloud Pub/Sub for data ingestion, Dataproc for processing, and Cloud SQL for storage
- D
Cloud Pub/Sub for data ingestion, Dataflow for processing, and Cloud Spanner for storage
- E
Cloud Storage for data ingestion, Dataflow for processing, and BigQuery for storage
Show answer and explanation
Correct answer: A
Explanation
To build a real-time analytics pipeline, you need tools optimized for streaming use cases. Cloud Pub/Sub is designed for low-latency ingestion of streaming data, Dataflow is a managed service for real-time data processing, and BigQuery provides scalable and fast storage for analytical workloads. This combination is the most suitable for the scenario described.
- A. Correct.
Correct. Cloud Pub/Sub is ideal for low-latency streaming data ingestion, Dataflow is well-suited for real-time processing, and BigQuery is optimized for storage and analysis.
- B. Incorrect.
Incorrect. Cloud Storage is not suitable for real-time ingestion because it is designed for batch processing rather than streaming data.
- C. Incorrect.
Incorrect. While Cloud Pub/Sub can be used for ingestion, Dataproc is not optimized for near real-time processing, and Cloud SQL is not ideal for storing analytical data at scale.
- D. Incorrect.
Incorrect. While Cloud Pub/Sub and Dataflow are suitable for ingestion and processing respectively, Cloud Spanner is a relational database and not the best choice for analytics storage compared to BigQuery.
- E. Incorrect.
Incorrect. Although Dataflow and BigQuery are suitable for processing and storage, Cloud Storage is not optimized for real-time ingestion of streaming data.