Google Professional Machine Learning Engineer Question 351
Select 3Google Cloud PlatformYou are deploying a machine learning model to production on Google Cloud, and the model's inference latency is higher than acceptable. After profiling, you discover that the model's complexity is causing excessive computational costs during serving. Which of the following actions can help reduce the latency while maintaining reasonable performance?
- A
Apply quantization to reduce the precision of weights and activations.
- B
Use a more complex model architecture to improve accuracy.
- C
Prune unnecessary neurons or weights in the model.
- D
Reduce the batch size during inference requests.
- E
Increase the number of training epochs to refine the model further.
Show answer and explanation
Correct answers: A, C, D
Explanation
To optimize a model for serving in production, techniques like quantization, pruning, and reducing the batch size can help simplify the model and reduce computational demands, thereby improving inference latency. It’s important to balance these optimizations with the need to maintain acceptable accuracy and performance levels.
- A. Correct.
Quantization is an effective technique to reduce model size and computational requirements by lowering the precision of weights and activations, which can significantly improve inference latency.
- B. Incorrect.
Using a more complex model architecture is counterproductive in this context as it would likely increase computational costs and latency further.
- C. Correct.
Pruning removes unnecessary neurons or weights, simplifying the model and reducing computational overhead during inference.
- D. Correct.
Reducing the batch size during inference decreases the number of inputs processed at a time, potentially improving latency for each individual request.
- E. Incorrect.
Increasing the number of training epochs is related to improving the model's accuracy during training but does not directly address inference latency in production.