Google Professional Data Engineer Question 40
Single answerGoogle Cloud PlatformA retail company is planning to migrate its existing on-premises data warehouse to Google Cloud. The current data warehouse supports daily batch processing for sales data and provides weekly reports to stakeholders. The company aims to transition into near real-time analytics in the next year to enable faster decision-making. Which architecture would best meet both the current and future business requirements?
- A
Deploy Google BigQuery for the data warehouse and use Cloud Composer to orchestrate daily batch processing workflows.
- B
Use Google BigQuery for the data warehouse and implement Pub/Sub and Dataflow for streaming data ingestion to support near real-time analytics.
- C
Set up a Google Cloud SQL instance as the data warehouse to handle structured data and use Cloud Functions to process data in real-time.
- D
Implement Google Dataproc clusters for batch processing and replace them with BigQuery when transitioning to near real-time analytics.
Show answer and explanation
Correct answer: B
Explanation
The correct answer is to use BigQuery, Pub/Sub, and Dataflow because this architecture supports both the current business requirement of daily batch processing and the future goal of enabling near real-time analytics. BigQuery is scalable and optimized for analytics, while Pub/Sub and Dataflow provide the necessary tools to handle streaming data ingestion and real-time processing. This approach ensures a seamless transition without requiring major architectural changes.
- A. Incorrect.
This option supports the current batch processing requirements but does not fully align with the company's future goal of transitioning to near real-time analytics. Cloud Composer is better suited for orchestrating workflows rather than enabling real-time streaming.
- B. Correct.
This option effectively supports both current and future business requirements. BigQuery can handle batch processing today, and the combination of Pub/Sub and Dataflow is ideal for enabling near real-time data ingestion and processing in the future.
- C. Incorrect.
While Cloud SQL is suitable for structured data, it is not designed for large-scale analytics use cases like a data warehouse. Additionally, Cloud Functions is not the optimal solution for streaming data ingestion.
- D. Incorrect.
While Dataproc can handle batch processing, transitioning to BigQuery later would require significant rearchitecting and would not be the most efficient or scalable approach to meet the company's needs for analytics.