Google Professional Data Engineer Question 39
Select 2Google Cloud PlatformYour company is developing an application that processes large amounts of data in real-time. The application is currently running on Google Cloud's BigQuery and Cloud Dataflow but may need to migrate to another cloud provider in the future due to business requirements. To ensure flexibility and portability, which two strategies should you implement when designing the application?
- A
Use open-source frameworks like Apache Beam for data processing pipelines.
- B
Tightly couple your application logic with Google Cloud-specific APIs for optimized performance.
- C
Design the storage layer to use open data formats such as Parquet or Avro.
- D
Use Google Cloud-specific services like BigQuery exclusively for all data storage and processing needs.
- E
Implement infrastructure as code (IaC) using tools like Terraform to manage cloud resources.
Show answer and explanation
Correct answers: A, C
Explanation
To design for flexibility and portability, it is critical to avoid vendor lock-in by using open-source frameworks and open data formats. These choices ensure your application can operate on multiple platforms and can be migrated with minimal rework, should business needs change. While tools like Terraform are valuable, they primarily assist with infrastructure management, not application-level portability.
- A. Correct.
Using open-source frameworks like Apache Beam allows you to design data processing pipelines that can run on multiple platforms, including Google Cloud Dataflow, Apache Flink, and Apache Spark. This enhances flexibility and portability.
- B. Incorrect.
Tightly coupling your application logic with Google Cloud-specific APIs can lead to vendor lock-in, reducing portability if you need to migrate to another cloud provider.
- C. Correct.
Using open data formats like Parquet or Avro ensures your data can be easily transferred and read by applications on other platforms or cloud providers, increasing portability.
- D. Incorrect.
Using cloud provider-specific services like BigQuery exclusively ties your solution to a specific provider, which can make migration to another cloud challenging.
- E. Incorrect.
While tools like Terraform are helpful for managing infrastructure, they do not directly contribute to application-level flexibility and portability as much as the use of open frameworks and data formats.