Google Professional Data Engineer Question 272
Select 3Google Cloud PlatformYou are designing a data processing pipeline on Google Cloud that needs to handle large-scale batch processing jobs. These jobs must run efficiently with minimal downtime and redundancy by utilizing multiple Google Cloud regions or zones. Which of the following strategies would best meet these requirements?
- A
Configure regional managed instance groups (MIGs) across multiple zones to distribute job workloads.
- B
Use a global Cloud Storage bucket to store job input and output data, ensuring accessibility across regions.
- C
Set up a zonal Compute Engine instance to run jobs and configure a backup instance in another zone manually.
- D
Leverage Dataproc workflow templates with region-specific clusters to distribute processing across regions.
- E
Enable multi-region BigQuery datasets for storing intermediate and final results to reduce latency across regions.
Show answer and explanation
Correct answers: A, B, E
Explanation
Running jobs efficiently in multiple regions or zones requires strategies that ensure redundancy, scalability, and accessibility. Regional MIGs distribute jobs across zones within a region, reducing the impact of zonal failures. A global Cloud Storage bucket ensures data is accessible across regions, avoiding duplication and providing high availability. Multi-region BigQuery datasets further optimize storage and retrieval for workflows that span regions. These approaches collectively ensure minimal downtime and optimal performance for large-scale data processing pipelines.
- A. Correct.
Correct. Regional MIGs allow for automatic distribution of workloads across zones in a region, improving redundancy and minimizing downtime.
- B. Correct.
Correct. Using a global Cloud Storage bucket ensures that job input and output data is accessible across multiple regions without needing duplication.
- C. Incorrect.
Incorrect. Zonal Compute Engine instances lack automatic failover, and manual backups are error-prone and do not scale well for large jobs.
- D. Incorrect.
Incorrect. While Dataproc workflow templates are useful for orchestration, they are not designed to automatically distribute workloads across regions, making it an inefficient choice for this scenario.
- E. Correct.
Correct. Multi-region BigQuery datasets improve accessibility and reduce latency for data storage and retrieval, making it an ideal choice for handling regional or multi-regional workloads.