Google Professional Machine Learning Engineer Question 350
Select 3Google Cloud PlatformYou are designing a machine learning model to predict customer churn for a subscription-based business. The model will be deployed to serve predictions in real-time with strict latency requirements. During testing, you notice that the model has high accuracy but is slow in serving predictions. Which of the following techniques can help optimize the model for production without significantly impacting accuracy?
- A
Quantizing the model to reduce the precision of weights and biases
- B
Using a smaller, less complex model architecture
- C
Increasing the batch size during inference
- D
Applying dropout during inference to reduce overfitting
- E
Using TensorFlow Serving or similar optimized serving infrastructure
Show answer and explanation
Correct answers: A, B, E
Explanation
Optimizing machine learning models for production requires a balance between accuracy and performance. Techniques like quantization and using smaller architectures can help reduce latency and improve efficiency without significant loss of accuracy. Additionally, using optimized serving infrastructure like TensorFlow Serving can further enhance the performance of real-time inference pipelines. However, techniques like increasing batch size or applying dropout during inference are either irrelevant or counterproductive for real-time scenarios.
- A. Correct.
Quantizing the model reduces the precision of weights and biases (e.g., from 32-bit floating-point to 8-bit integers), which can significantly improve inference speed and reduce memory usage without drastically reducing accuracy.
- B. Correct.
Using a smaller, less complex model architecture can reduce computational overhead during inference, leading to faster predictions. This is especially useful when strict latency requirements exist.
- C. Incorrect.
Increasing the batch size during inference is typically used to improve throughput but does not help with reducing latency for real-time predictions, as it can delay the prediction for individual inputs.
- D. Incorrect.
Applying dropout during inference is not a valid optimization technique, as dropout is typically used during training to prevent overfitting and is turned off during inference.
- E. Correct.
Using TensorFlow Serving or similar optimized serving infrastructure can improve the efficiency of serving predictions by leveraging optimizations such as model caching and hardware acceleration.