Google Professional Data Engineer Question 228
Single answerGoogle Cloud PlatformYour organization runs business-critical data processing pipelines using Google Cloud Dataflow. During periodic spikes in workload, these pipelines experience delays due to resource limitations, leading to missed SLAs. You need to ensure that sufficient resources are available during these spikes without over-provisioning during normal operations. What should you do?
- A
Enable autoscaling in your Dataflow pipeline configurations.
- B
Manually provision additional VM instances during peak workloads.
- C
Switch to preemptible VMs to reduce the cost of the pipelines.
- D
Configure Dataflow to use a fixed number of workers to ensure resource predictability.
Show answer and explanation
Correct answer: A
Explanation
Enabling autoscaling in Dataflow is the best approach to ensure that enough resources are available for business-critical data processes during workload spikes. Autoscaling dynamically adjusts the number of workers based on the workload, ensuring that SLAs are met while avoiding unnecessary costs during normal operations.
- A. Correct.
Enabling autoscaling in Dataflow allows the service to automatically add or remove workers based on the current workload. This ensures that resources are scaled up during spikes and scaled down during normal operations, optimizing both performance and cost.
- B. Incorrect.
Manually provisioning resources can be error-prone, time-consuming, and inefficient, especially during sudden workload spikes. It does not address the need for dynamic scaling.
- C. Incorrect.
Preemptible VMs reduce costs but are subject to termination at any time, which can disrupt business-critical pipelines. This approach is unsuitable for ensuring resource availability during spikes.
- D. Incorrect.
Using a fixed number of workers does not account for workload fluctuations, leading to either resource shortages during spikes or over-provisioning during normal operations.