Google Professional Data Engineer Question 123
Select 2Google Cloud PlatformYour company processes large volumes of IoT sensor data in real-time, requiring low-latency writes and the ability to query recent data efficiently. Historical data, however, is rarely accessed but must be retained for compliance for seven years. Which combination of Google Cloud storage services should you use to meet these requirements?
- A
Cloud Bigtable for real-time data and Cloud Storage for historical data
- B
Cloud SQL for real-time data and Cloud Storage for historical data
- C
Cloud Spanner for real-time data and BigQuery for historical data
- D
Cloud Bigtable for real-time data and BigQuery for historical data
- E
Cloud Firestore for real-time data and BigQuery for historical data
Show answer and explanation
Correct answers: A, D
Explanation
For real-time IoT data, a storage system optimized for high-throughput and low-latency writes, such as Cloud Bigtable, is ideal. For historical data that is infrequently accessed but retained for compliance, using a cost-effective and scalable solution like Cloud Storage or BigQuery is appropriate. The combination of Cloud Bigtable and Cloud Storage or Cloud Bigtable and BigQuery effectively balances these requirements, offering the best performance and cost-efficiency.
- A. Correct.
Cloud Bigtable is optimized for low-latency writes and high-throughput operations, making it suitable for real-time IoT data. Cloud Storage is a cost-effective option for retaining infrequently accessed historical data.
- B. Incorrect.
Cloud SQL is not designed for high-throughput, low-latency data ingestion, making it unsuitable for real-time IoT data. However, Cloud Storage could work for historical data.
- C. Incorrect.
Cloud Spanner is highly scalable but is designed for transactional workloads rather than high-throughput IoT streaming. BigQuery is good for analytical queries but using it for real-time ingestion is not optimal.
- D. Correct.
Cloud Bigtable suits real-time data ingestion due to its low-latency, high-throughput capabilities. BigQuery is excellent for analyzing large-scale historical data with advanced querying capabilities.
- E. Incorrect.
Cloud Firestore is a NoSQL database suitable for application development but not ideal for high-throughput IoT ingestion. BigQuery is suitable for analyzing historical data, but it doesn't suffice for real-time ingestion.