Google Professional Machine Learning Engineer Question 321
Select 4Google Cloud PlatformYou are deploying a machine learning model for online inference on Google Cloud. The model is expected to handle unpredictable traffic spikes and low-latency requirements. Which of the following considerations should you prioritize for scaling online model serving?
- A
Use Google Kubernetes Engine (GKE) with horizontal pod autoscaling enabled.
- B
Deploy the model on a single Compute Engine instance with a fixed machine type.
- C
Implement a load balancer to distribute traffic across multiple model replicas.
- D
Pre-warm the serving infrastructure to handle sudden spikes in traffic.
- E
Use AI Platform (Vertex AI) Predictions with auto-scaling enabled.
Show answer and explanation
Correct answers: A, C, D, E
Explanation
Scaling online model serving involves ensuring that the infrastructure can dynamically adjust to traffic patterns while maintaining low latency and high availability. Using GKE with horizontal pod autoscaling, Vertex AI Predictions with auto-scaling, load balancing, and pre-warming infrastructure are all effective strategies to handle these requirements. However, using a single fixed Compute Engine instance lacks scalability and redundancy, making it unsuitable for this scenario.
- A. Correct.
Correct: Google Kubernetes Engine (GKE) with horizontal pod autoscaling is a robust choice for handling unpredictable traffic spikes since it can dynamically adjust the number of pods based on demand.
- B. Incorrect.
Incorrect: Deploying the model on a single Compute Engine instance with a fixed machine type does not provide scalability or redundancy. It is unsuitable for handling traffic spikes or ensuring low-latency responses.
- C. Correct.
Correct: A load balancer is essential to distribute incoming requests across multiple model replicas, ensuring better performance and reliability during traffic spikes.
- D. Correct.
Correct: Pre-warming the serving infrastructure ensures that your system is ready to handle sudden increases in traffic without delays caused by cold starts.
- E. Correct.
Correct: AI Platform (Vertex AI) Predictions with auto-scaling enabled can automatically adjust resources based on traffic patterns, making it ideal for handling unpredictable traffic and maintaining low latency.