Google Professional Machine Learning Engineer Question 353
Select 3Google Cloud PlatformYou are designing a machine learning model for real-time predictions in a production environment with strict latency requirements. During testing, you notice that the model's inference time is too high and its performance exceeds what is required for the use case. Which of the following techniques could you use to simplify the model and improve inference latency while maintaining acceptable accuracy?
- A
Reduce model complexity by pruning unnecessary parameters or neurons.
- B
Quantize the model to use lower-precision data types such as INT8 instead of FP32.
- C
Increase the model's training data size to improve its generalization.
- D
Use knowledge distillation to train a smaller model using the outputs of the original model.
- E
Deploy the model using a larger VM instance with more GPUs to reduce latency.
Show answer and explanation
Correct answers: A, B, D
Explanation
To meet strict latency requirements for real-time predictions, you can simplify the model using techniques like pruning, quantization, or knowledge distillation. These approaches reduce the computational complexity of the model, which directly improves inference latency while maintaining acceptable accuracy. Increasing training data or scaling up hardware may improve performance in other areas but does not simplify the model, which is the key requirement in this scenario.
- A. Correct.
Reducing model complexity by pruning unnecessary parameters or neurons is a valid simplification technique that can reduce inference latency with minimal impact on accuracy when done carefully.
- B. Correct.
Quantizing the model to use lower-precision data types, such as INT8, can significantly improve inference speed without a significant loss in accuracy, especially for use cases with strict latency requirements.
- C. Incorrect.
Increasing the model's training data size is focused on improving model generalization during training but does not directly address inference latency or simplify the model.
- D. Correct.
Knowledge distillation involves training a smaller 'student' model to mimic the behavior of a larger 'teacher' model. This can effectively reduce model size and inference time while maintaining acceptable accuracy.
- E. Incorrect.
Deploying the model on a larger VM instance with more GPUs can reduce latency but does not simplify the model itself, which is the focus of the given scenario.