Google Professional Machine Learning Engineer Question 339
Single answerGoogle Cloud PlatformYou have deployed a machine learning model on Google Cloud's Vertex AI Prediction service for real-time predictions. The traffic to your model's endpoint varies significantly throughout the day, with spikes during business hours. To handle this varying throughput efficiently, how can you scale the serving backend of your model?
- A
Manually monitor the traffic and adjust the number of nodes in the model's endpoint as needed.
- B
Configure auto-scaling for the model endpoint based on request-per-second (RPS) metrics.
- C
Deploy the model to a Kubernetes cluster and use Kubernetes Horizontal Pod Autoscaler (HPA).
- D
Use a static number of nodes for the endpoint and overprovision resources to handle peak traffic.
Show answer and explanation
Correct answer: B
Explanation
Vertex AI Prediction provides built-in auto-scaling capabilities that allow the serving backend to dynamically adjust the number of nodes based on real-time traffic metrics, such as requests-per-second (RPS). This approach ensures efficient resource utilization and cost management while maintaining low latency during traffic spikes. Manual adjustments and static provisioning are less efficient and not recommended for dynamic traffic scenarios.
- A. Incorrect.
Manually monitoring and adjusting the number of nodes is inefficient and error-prone, especially for applications with dynamic traffic patterns.
- B. Correct.
Configuring auto-scaling for the model endpoint based on RPS is the correct approach. Vertex AI Prediction supports auto-scaling, which optimizes resource usage by dynamically adjusting the number of nodes based on traffic.
- C. Incorrect.
While Kubernetes HPA can handle auto-scaling, it is not the most efficient or recommended way to scale serving backends in Vertex AI Prediction, as the service provides built-in auto-scaling capabilities tailored for ML models.
- D. Incorrect.
Using a static number of nodes and overprovisioning resources leads to inefficient resource utilization and higher costs, especially during periods of low traffic.