Google Professional Data Engineer Question 75
Select 3Google Cloud PlatformYou are working for a retail company that collects data from multiple sources like transactional databases, clickstream logs, and third-party APIs. You need to design a data transformation logic in Google Cloud to clean, enrich, and aggregate this data for downstream analytics in BigQuery. Which of the following approaches should you consider?
- A
Use Dataflow with Apache Beam to define and execute complex transformation pipelines.
- B
Use Cloud Functions to trigger and process data transformations in real time.
- C
Load raw data into BigQuery and define SQL-based transformations using views.
- D
Use Dataproc to run Spark jobs for batch transformations on large datasets.
- E
Use Cloud Spanner to perform data transformations within the database.
Show answer and explanation
Correct answers: A, C, D
Explanation
The correct choices involve tools that are optimized for data transformation in Google Cloud. Dataflow is ideal for scalable and real-time or batch pipelines, BigQuery with SQL is effective for analytics-focused transformations, and Dataproc with Spark is suitable for batch transformations on large datasets. Cloud Functions and Cloud Spanner, while useful for other purposes, are not suited for the described use case.
- A. Correct.
Correct: Dataflow with Apache Beam is a scalable and flexible tool for defining and executing complex transformation pipelines, making it ideal for cleaning, enriching, and aggregating data.
- B. Incorrect.
Incorrect: While Cloud Functions can process data in real time, they are not well-suited for complex or large-scale data transformation pipelines.
- C. Correct.
Correct: Loading raw data into BigQuery and using SQL-based transformations through views is a common approach for data transformation in Google Cloud, especially for analytics use cases.
- D. Correct.
Correct: Dataproc with Spark is a good choice for batch processing and transformations on large datasets and provides flexibility in handling different data formats.
- E. Incorrect.
Incorrect: Cloud Spanner is a globally distributed database optimized for transactions, not for complex data transformations.