gcp data engineer Practice Exam 2025: Latest Questions

A retail company stores raw clickstream JSON files in Cloud Storage and wants analysts to query them in BigQuery without managing a loading pipeline. The schema may evolve (new optional fields). What is the recommended approach?

You need to ingest events from thousands of IoT devices with occasional spikes. The system must buffer reliably and allow multiple downstream consumers (stream processing for real-time alerts and batch processing for analytics). Which Google Cloud service should be the primary ingestion layer?

A dataset contains sensitive customer PII in BigQuery. Analysts should be able to query aggregate metrics but must not see raw PII columns. What is the best practice to enforce this with minimal application changes?

A Dataflow batch job processes files from Cloud Storage. You want to run it daily without managing servers and with retry handling. What should you use?

Your company runs a streaming Dataflow pipeline reading from Pub/Sub and writing to BigQuery. During traffic surges, BigQuery streaming inserts occasionally fail due to quota/throughput constraints. You must minimize data loss and keep processing near real time. What is the recommended design change?

A marketing team needs to run federated queries in BigQuery on data stored in Cloud Storage and in an external SaaS system. The organization requires central governance, access control, and metadata management across these sources. Which approach best meets the requirement?

You have a BigQuery table partitioned by event_date. Queries are slow and expensive because analysts frequently filter by event_date AND customer_id. What is the best optimization to improve performance with minimal changes?

A Dataflow pipeline writes transformed data into BigQuery. Analysts report duplicate rows appearing after worker restarts. The pipeline currently uses at-least-once delivery semantics. What is the best way to make results effectively exactly-once for analytics?

You are designing a lakehouse on Google Cloud. Data lands as Parquet in Cloud Storage, and multiple teams need fine-grained access controls and consistent governance while querying from BigQuery. You must avoid copying data into BigQuery-managed storage. Which design best meets these requirements?

A regulated enterprise must ensure that all data pipeline changes (Dataflow templates, BigQuery schema changes, and orchestration) are auditable, promote through environments (dev/test/prod), and can be rolled back quickly. What is the best approach?

A team writes daily Parquet files to Cloud Storage. Analysts query them from BigQuery but frequently get schema mismatch errors because upstream teams occasionally add optional fields. The team wants queries to succeed without manual schema updates while keeping the data in Cloud Storage. What should they do?

You need to ingest data from on-premises PostgreSQL into BigQuery with low operational overhead. The business requires near real-time replication and the ability to backfill tables. Which Google Cloud service is the best fit?

A data engineering team runs scheduled BigQuery queries that sometimes exceed their time window and overlap with the next run, causing duplicate writes to destination tables. They want to prevent concurrent executions of the same scheduled workload. What is the recommended approach?

You are designing a streaming pipeline that ingests IoT events into Pub/Sub and processes them with Dataflow. The pipeline must handle out-of-order events, compute 5-minute windowed aggregates, and ensure late events up to 30 minutes are included in the correct window. Which Dataflow concept should you use?

A BigQuery dataset stores sensitive columns (e.g., SSN). Analysts should be able to query aggregates and join on non-sensitive keys, but only a small security group can see raw SSNs. You want the simplest approach without duplicating tables. What should you implement?

A Dataflow job writes processed records to BigQuery. You observe intermittent failures with errors indicating too many streaming inserts and occasional duplicate rows when the job retries. You need a solution that improves reliability and minimizes duplicates. What should you do?

A company stores raw clickstream logs in Cloud Storage and wants to curate them into a trusted analytics dataset. They need to enforce data quality rules (e.g., required fields, value ranges) and capture rule failures for audit. The solution should be managed and integrated with BigQuery. What should they use?

A pipeline processes files dropped into a Cloud Storage bucket. Some files are sometimes partially uploaded when processing begins, causing parse errors. You want a robust design that prevents processing incomplete files and requires minimal custom code. What should you do?

A Dataflow batch job reads from Cloud Storage and writes to BigQuery. It succeeds for small runs but fails at scale due to worker disk exhaustion and frequent shuffle spills. You need to improve performance and stability with minimal changes to business logic. What should you do?

You manage a multi-project data platform. Raw data lands in a central project, while curated BigQuery datasets live in separate domain projects. You need to allow Dataflow jobs running in a processing project to read from the raw bucket and write to domain BigQuery datasets, while keeping least privilege and avoiding long-lived keys. What is the best approach?

A retail company uses BigQuery for analytics. Analysts run exploratory queries that frequently scan large tables and occasionally exceed per-user query limits. The company wants to reduce the risk of runaway costs and improve fairness across teams without blocking legitimate workloads. What should you do?

You are ingesting JSON events from a Pub/Sub topic into BigQuery using a streaming pipeline. Recently, some rows appear duplicated in BigQuery during brief subscriber restarts. The business requires exactly-once results for downstream reporting. What is the recommended approach?

A team needs to share a curated BigQuery dataset with a partner organization. The partner should be able to query the data but must not be able to copy underlying tables into their own project or export the data to Cloud Storage. What is the best solution?

You have a Cloud Storage bucket containing Parquet files partitioned by date in the path (e.g., gs://bucket/events/dt=2026-01-01/...). Analysts want to query the files from BigQuery without loading them first and want partition pruning by date. What should you implement?

A data platform runs multiple Dataflow batch pipelines nightly. After a recent change, one pipeline intermittently fails with 'quota exceeded' errors when writing to BigQuery and causes downstream pipelines to start with partial data. You need to improve reliability and ensure downstream jobs only run after successful completion with clear failure signaling. What should you do?