Google Professional Machine Learning Engineer Question 319
Select 3Google Cloud PlatformYour organization has deployed a machine learning model for real-time predictions on Google Cloud. The model is hosted on a Vertex AI endpoint with auto-scaling enabled. As user traffic increases significantly during peak hours, you notice increased latency and occasional timeout errors. What steps should you take to ensure the system scales efficiently and maintains low latency?
- A
Increase the maximum number of replicas in the auto-scaling configuration for the endpoint.
- B
Enable regional endpoint replication to distribute traffic across multiple regions.
- C
Reduce the minimum number of replicas in the auto-scaling configuration to conserve costs during low-traffic periods.
- D
Optimize the model code and container serving environment for faster inference times.
- E
Use a global external HTTP(S) load balancer in front of the endpoint.
Show answer and explanation
Correct answers: A, D, E
Explanation
To scale online model serving effectively, you need to ensure the system can handle increased traffic while minimizing latency. Increasing the maximum number of replicas allows auto-scaling to provision sufficient resources during peak loads. Optimizing the model and serving environment reduces inference latency. Finally, a global HTTP(S) load balancer helps distribute traffic efficiently, preventing bottlenecks and ensuring high availability. Regional endpoint replication is more suited for cross-region availability rather than addressing latency in a single region, and reducing the minimum replicas can negatively impact responsiveness during scaling events.
- A. Correct.
Correct: Increasing the maximum number of replicas ensures that the system can handle higher traffic during peak hours by provisioning more resources.
- B. Incorrect.
Incorrect: While regional replication can improve availability across regions, it does not directly address latency caused by local scaling issues.
- C. Incorrect.
Incorrect: Reducing the minimum number of replicas may save costs, but it can lead to slower scaling responses during traffic spikes, exacerbating latency issues.
- D. Correct.
Correct: Optimizing the model code and serving environment reduces inference times, directly addressing the latency issue.
- E. Correct.
Correct: A global external HTTP(S) load balancer helps route traffic more efficiently and evenly distributes requests to the available replicas, reducing bottlenecks and improving scalability.