Google Professional Machine Learning Engineer Question 320
Select 2Google Cloud PlatformYour team has deployed a machine learning model for real-time predictions using Google Cloud's Vertex AI Prediction service. The service is experiencing inconsistent latency during peak traffic periods. You need to ensure the serving infrastructure scales effectively to handle fluctuating demand while maintaining low latency. Which actions should you take?
- A
Enable autoscaling for your model endpoint in Vertex AI Prediction.
- B
Increase the instance type used for model serving to a more powerful machine type.
- C
Implement a caching layer to store frequently requested predictions.
- D
Set up a load balancer in front of your model endpoint to distribute traffic evenly.
- E
Configure the minimum and maximum replica settings for the model endpoint.
Show answer and explanation
Correct answers: A, E
Explanation
To handle scaling for online model serving in Vertex AI Prediction, enabling autoscaling allows the service to dynamically adjust resources based on traffic. Configuring minimum and maximum replica counts ensures the autoscaler has appropriate boundaries to scale within, guaranteeing both cost efficiency and performance. Other options, such as increasing machine type or implementing caching, may help with performance but do not directly address scalability with fluctuating demand.
- A. Correct.
Enabling autoscaling allows the infrastructure to automatically adjust the number of replicas based on traffic demand, which is crucial for handling fluctuating loads.
- B. Incorrect.
Simply increasing the instance type might improve performance temporarily but does not address scalability in a cost-effective way, especially during fluctuating demand.
- C. Incorrect.
A caching layer might help reduce latency for repeated predictions but doesn't directly address the need to scale the underlying model serving infrastructure.
- D. Incorrect.
While a load balancer can help distribute traffic, Vertex AI Prediction endpoints already handle load distribution internally, so this step is unnecessary.
- E. Correct.
Configuring the minimum and maximum replica settings ensures that the autoscaler has boundaries to scale within, allowing it to handle both low and high traffic effectively.