Google Professional Data Engineer Question 126
Single answerGoogle Cloud PlatformYou are tasked with designing a data pipeline that processes billions of records daily in Google Cloud. Your team wants to identify patterns in data access to optimize the storage and querying strategy. Which approach should you take to analyze data access patterns effectively?
- A
Use Cloud Audit Logs to monitor storage bucket access and analyze the logs for trends.
- B
Leverage BigQuery's INFORMATION_SCHEMA to query historical query statistics and access patterns.
- C
Enable Stackdriver Trace to capture request latencies and infer access patterns.
- D
Set up Dataflow to continuously process and log access requests in real-time.
Show answer and explanation
Correct answer: B
Explanation
BigQuery’s INFORMATION_SCHEMA is specifically designed to provide metadata about query execution and table access. It allows you to identify trends in data usage, such as which tables are accessed most frequently, query performance statistics, and user activity. This information is critical for optimizing storage and querying strategies in a data pipeline.
- A. Incorrect.
Cloud Audit Logs provide detailed information about access to resources but are primarily focused on security and compliance, not detailed access pattern analysis.
- B. Correct.
BigQuery's INFORMATION_SCHEMA provides metadata about query usage, including frequency, user activity, and table access patterns. This makes it an ideal choice for analyzing access patterns in a data pipeline.
- C. Incorrect.
Stackdriver Trace is designed to monitor latency and performance for applications, not to analyze data access patterns.
- D. Incorrect.
While Dataflow can process data streams, it is not a direct tool for analyzing historical access patterns or trends.