Google Professional Data Engineer Question 121
Single answerGoogle Cloud PlatformYour organization stores large amounts of semi-structured data in Google Cloud Storage. You need to query this data regularly using SQL without moving it to another service. Which solution should you use to achieve this?
- A
Enable BigQuery's external table feature to query the data directly from Cloud Storage
- B
Use Dataproc to load the data into HDFS and query it using Hive
- C
Export the data to BigQuery tables and query it from there
- D
Use Dataflow to process the data and then write SQL queries on the processed output
Show answer and explanation
Correct answer: A
Explanation
BigQuery's external table feature is the most efficient and cost-effective solution for querying semi-structured data directly in Cloud Storage using SQL. It avoids unnecessary data movement, supports querying in place, and integrates seamlessly with Cloud Storage. The other options introduce unnecessary complexity or contradict the requirement to avoid moving the data.
- A. Correct.
Correct. BigQuery's external table feature allows you to query data directly from Cloud Storage without having to move or load it into BigQuery, which is ideal for this use case.
- B. Incorrect.
Incorrect. While Dataproc can process data using tools like Hive, this setup is more complex and requires additional infrastructure, making it less suitable for directly querying semi-structured data in Cloud Storage.
- C. Incorrect.
Incorrect. Exporting the data to BigQuery tables involves moving the data into BigQuery, which contradicts the requirement to query the data directly without moving it.
- D. Incorrect.
Incorrect. Dataflow is a stream and batch processing service. While it can process the data, it is not designed for directly querying data using SQL.