Google Professional Machine Learning Engineer Question 96
Select 2Google Cloud PlatformYou are designing a data processing pipeline for a financial analytics company that needs to process large volumes of transactional data daily. The pipeline must support real-time transactional consistency, low-latency processing, and the ability to run large-scale batch analytics. Which combination of Google Cloud and open-source technologies should you choose?
- A
Use Cloud Spanner to handle transactional data and Apache Spark for batch analytics.
- B
Use Cloud SQL for transactional data and Apache Hadoop for batch analytics.
- C
Use Apache Spark for both transactional data and batch analytics.
- D
Use Cloud Spanner for transactional data and Cloud Dataflow for large-scale batch analytics.
- E
Use BigQuery for transactional data and Apache Spark for batch analytics.
Show answer and explanation
Correct answers: A, D
Explanation
The pipeline requires a combination of technologies that can handle real-time transactional consistency and large-scale batch analytics. Cloud Spanner is ideal for transactional data due to its strong consistency and low latency. For batch analytics, both Apache Spark and Cloud Dataflow are suitable options. Thus, the best combinations are Cloud Spanner with Apache Spark and Cloud Spanner with Cloud Dataflow.
- A. Correct.
Cloud Spanner is a globally distributed database that provides strong transactional consistency and low-latency reads/writes. Apache Spark is well-suited for large-scale batch analytics, making this a valid combination.
- B. Incorrect.
Cloud SQL supports transactional workloads, but it is not optimized for large-scale distributed systems or high-volume batch analytics. Apache Hadoop is suitable for batch analytics but not ideal for real-time transactional consistency, making this option less suitable.
- C. Incorrect.
Apache Spark is designed for distributed data processing and analytics but does not handle transactional data with strong consistency, making this choice unsuitable for the transactional component.
- D. Correct.
Cloud Spanner provides the required transactional consistency and low latency for real-time data, while Cloud Dataflow is a managed service for large-scale batch and stream analytics. This combination satisfies both requirements for the pipeline.
- E. Incorrect.
BigQuery is optimized for analytical workloads, not transactional processing. It is not suitable for real-time transactional consistency, so this option is incorrect.