Google Professional Data Engineer Question 223
Select 3Google Cloud PlatformYour organization processes large amounts of data daily for batch analytics. The data is stored in Google Cloud Storage and processed using BigQuery. However, your manager has requested that you minimize costs while ensuring performance is not heavily impacted. Which of the following strategies should you implement?
- A
Use BigQuery table partitions and clustering to reduce scanned data size during queries.
- B
Switch from BigQuery on-demand pricing to flat-rate pricing without evaluating query patterns.
- C
Compress the data in Google Cloud Storage using an efficient format like Parquet or Avro.
- D
Use BigQuery Materialized Views for frequently queried aggregated data.
- E
Enable detailed Stackdriver (Cloud Monitoring) logs for all BigQuery queries to analyze costs.
Show answer and explanation
Correct answers: A, C, D
Explanation
Minimizing costs in a data processing workflow requires strategies that reduce both storage and query costs without significantly impacting performance. Partitioning and clustering reduce the data scanned during queries, while data compression lowers storage costs. Materialized Views help optimize queries by precomputing results. On the other hand, switching to flat-rate pricing without analysis and enabling detailed logs could increase costs unnecessarily.
- A. Correct.
Using table partitions and clustering helps minimize the amount of data scanned during queries, which directly reduces BigQuery costs while maintaining performance.
- B. Incorrect.
Switching to flat-rate pricing without evaluating query patterns could lead to higher costs if your usage doesn't justify the flat-rate commitment.
- C. Correct.
Compressing data using efficient formats like Parquet or Avro reduces storage costs and improves query performance by reducing the amount of data processed.
- D. Correct.
Using Materialized Views for frequently queried aggregated data reduces the need to recompute results, lowering query costs and improving performance.
- E. Incorrect.
Enabling detailed logs for all queries will increase costs for logging and monitoring without directly contributing to minimizing query or storage costs.