Google Professional Data Engineer Question 159
Single answerGoogle Cloud PlatformYour organization is building a data pipeline to process and analyze streaming data from IoT sensors in real time. The pipeline must handle high-throughput data, ensure exactly-once processing semantics, and allow for windowed computations such as aggregating sensor readings every minute. Which Google Cloud service is the most suitable for achieving these requirements?
- A
Cloud Dataflow
- B
BigQuery
- C
Cloud Dataproc
- D
Pub/Sub
Show answer and explanation
Correct answer: A
Explanation
The scenario requires a system capable of handling real-time streaming data, ensuring exactly-once processing semantics, and performing windowed computations. Cloud Dataflow is the most suitable service because it is designed for stream and batch data processing and supports features like windowing and exactly-once processing. Other options either lack these capabilities or are unsuitable for real-time stream processing.
- A. Correct.
Cloud Dataflow is designed for real-time stream processing and batch processing. It supports windowed computations, exactly-once processing semantics, and scales to handle high-throughput data, making it the most suitable choice for the scenario.
- B. Incorrect.
BigQuery is a data warehouse service optimized for analytical queries on large datasets. While it supports streaming inserts, it is not designed for real-time stream processing or windowed computations.
- C. Incorrect.
Cloud Dataproc is a managed Hadoop and Spark service, suitable for batch processing and complex data transformations, but it is not optimized for real-time stream processing or handling exactly-once semantics.
- D. Incorrect.
Pub/Sub is a messaging service that can ingest streaming data but does not directly support real-time stream processing or windowed computations. It is often used in conjunction with services like Cloud Dataflow.