Google Professional Data Engineer Question 242
Select 2Google Cloud PlatformYour organization needs to implement a data processing workflow in Google Cloud to support two business units with distinct requirements. Business Unit A requires a low-latency data pipeline for real-time analytics, while Business Unit B requires a cost-effective solution for batch processing large datasets. How should you organize the workloads to meet these requirements?
- A
Use Dataflow with streaming mode for Business Unit A and Dataflow with batch mode for Business Unit B.
- B
Use BigQuery for Business Unit A and Cloud Dataproc for Business Unit B.
- C
Use Pub/Sub for Business Unit A and Cloud Storage for Business Unit B.
- D
Use Dataflow with streaming mode for both Business Units.
- E
Use separate projects for each Business Unit, with Dataflow for Business Unit A and Cloud Dataproc for Business Unit B.
Show answer and explanation
Correct answers: A, E
Explanation
The correct answers involve using the appropriate tools and organizational strategies to meet the distinct needs of the business units. Dataflow is an ideal choice for real-time analytics (streaming mode) for Business Unit A, while batch mode or Cloud Dataproc can handle cost-effective batch processing for Business Unit B. Using separate projects ensures resource isolation, clearer billing, and better governance.
- A. Correct.
This is a correct option because Dataflow in streaming mode caters to real-time processing needs for Business Unit A, and Dataflow in batch mode is suitable for Business Unit B's cost-effective batch processing requirement.
- B. Incorrect.
This option is incorrect because BigQuery is not designed for low-latency real-time streaming, and Cloud Dataproc might not be the most cost-effective for large batch jobs compared to other options like Dataflow.
- C. Incorrect.
This option is incorrect because Pub/Sub alone cannot handle the processing of data for analytics, and Cloud Storage does not provide data processing capabilities.
- D. Incorrect.
This option is incorrect because using streaming mode for both Business Units is not cost-effective for Business Unit B, which requires batch processing.
- E. Correct.
This is a correct option because using separate projects allows you to isolate resources and permissions for each Business Unit while also tailoring the solutions to meet their individual requirements (Dataflow for streaming in Business Unit A, and Cloud Dataproc for batch processing in Business Unit B).