Google Professional Data Engineer Question 183
Single answerGoogle Cloud PlatformYou are designing a data pipeline for an e-commerce application that processes millions of transactions daily. To improve query performance, you want to precalculate certain fields that are frequently used in downstream analytics, such as the total transaction amount (unit price multiplied by quantity). Which of the following is the most appropriate way to implement precalculated fields in your Google Cloud data pipeline?
- A
Use BigQuery views to dynamically calculate the fields during query execution.
- B
Add the precalculated fields during ETL processing and store them in BigQuery.
- C
Use Dataflow to calculate the fields dynamically during query execution.
- D
Rely on client-side code to calculate the fields after fetching raw data from BigQuery.
Show answer and explanation
Correct answer: B
Explanation
Precalculating fields during the ETL process and storing them in BigQuery aligns with best practices for optimizing query performance in Google Cloud. This approach ensures that frequently used fields, such as total transaction amount, are precomputed and readily available in the dataset. This minimizes computational overhead during query execution and enhances the efficiency of downstream analytics processes.
- A. Incorrect.
BigQuery views calculate fields dynamically at query time, which can be computationally expensive and does not align with the goal of precalculation.
- B. Correct.
Adding the precalculated fields during ETL processing and storing them in BigQuery ensures that the fields are already computed and available for downstream analytics, improving query performance.
- C. Incorrect.
Using Dataflow to calculate fields dynamically during query execution is not efficient for precalculation since the fields would still need to be calculated every time they are queried.
- D. Incorrect.
Relying on client-side code to calculate fields after fetching raw data from BigQuery is inefficient and goes against the principle of optimizing query performance at the storage or query layer.