Google Professional Data Engineer Question 224
Select 2Google Cloud PlatformYour company processes large volumes of IoT sensor data daily. The data must be ingested, stored, and analyzed for real-time alerts and long-term trend analysis. The real-time alerts require data processing within seconds, while trend analysis can be processed in batches once per day. To minimize costs while meeting these business needs, which combination of GCP services should you use?
- A
Use Cloud Pub/Sub for real-time ingestion and Cloud Dataflow for real-time processing.
- B
Use BigQuery for both real-time alerts and long-term trend analysis.
- C
Use Cloud Pub/Sub for real-time ingestion, Cloud Dataflow for real-time processing, and BigQuery for trend analysis.
- D
Use Cloud Dataproc for both real-time alerts and batch processing of trend analysis.
- E
Use Cloud Storage for real-time ingestion and Cloud Dataflow for both real-time alerts and batch trend analysis.
Show answer and explanation
Correct answers: A, C
Explanation
To minimize costs while meeting the business need for both real-time alerts and long-term trend analysis, a combination of services is required. Cloud Pub/Sub and Cloud Dataflow are optimal and cost-effective for real-time ingestion and processing, while BigQuery is ideal for cost-efficient, large-scale analytics for trend analysis. This approach ensures that each business requirement is met with the most appropriate and cost-effective GCP services.
- A. Correct.
Cloud Pub/Sub and Cloud Dataflow are cost-effective options for real-time ingestion and processing, ensuring data is processed within seconds for real-time alerts. However, this option alone does not address long-term trend analysis.
- B. Incorrect.
BigQuery is an excellent choice for analytics, particularly for long-term trend analysis, but it is not optimized for real-time processing due to latency and cost considerations.
- C. Correct.
This combination balances cost and performance by using Cloud Pub/Sub and Cloud Dataflow for real-time ingestion and processing, while utilizing BigQuery for cost-effective trend analysis.
- D. Incorrect.
Cloud Dataproc is generally used for batch data processing and is not optimized for real-time alerts, making it a less cost-effective option for this use case.
- E. Incorrect.
Cloud Storage is not designed for real-time ingestion, and using Cloud Dataflow for both real-time and batch processing can lead to higher costs compared to using the more appropriate services like BigQuery for trend analysis.