Google Professional Machine Learning Engineer Question 316
Select 3Google Cloud PlatformYou are a Machine Learning Engineer at a retail company, and you have deployed two versions of a recommendation model: Model A and Model B. You want to perform A/B testing to evaluate which model provides better performance in terms of click-through rate (CTR). You decide to use Google Cloud's AI Platform Prediction to manage the tests. Which of the following steps should you take to correctly implement A/B testing in this scenario?
- A
Deploy both Model A and Model B to separate endpoints in AI Platform Prediction and route traffic using a load balancer.
- B
Deploy both Model A and Model B to the same endpoint in AI Platform Prediction and configure traffic splitting.
- C
Use AI Platform Prediction’s traffic-splitting feature to allocate a percentage of incoming requests to each model.
- D
Ensure that the evaluation metric (CTR) is tracked for each version of the model during the testing period.
- E
Manually alternate requests between Model A and Model B to ensure equal traffic distribution.
Show answer and explanation
Correct answers: B, C, D
Explanation
In Google Cloud's AI Platform Prediction, A/B testing is implemented by deploying both models to the same endpoint and utilizing the traffic-splitting feature to distribute a percentage of traffic to each model. It's also important to track the evaluation metric (CTR) to compare model performance effectively. Using separate endpoints or manually alternating traffic is not recommended as it adds unnecessary complexity and potential for errors.
- A. Incorrect.
Incorrect. Deploying models to separate endpoints and routing traffic using a load balancer is not the recommended approach for A/B testing in Google Cloud. AI Platform Prediction provides built-in traffic-splitting functionality for this purpose.
- B. Correct.
Correct. Deploying both models to the same endpoint and configuring traffic splitting is a best practice for A/B testing in AI Platform Prediction. This allows you to manage traffic distribution efficiently.
- C. Correct.
Correct. AI Platform Prediction has a traffic-splitting feature that lets you allocate a specific percentage of incoming requests to each model, which is essential for A/B testing.
- D. Correct.
Correct. Tracking the evaluation metric (CTR) for each model version is crucial to determine which model performs better during the A/B test.
- E. Incorrect.
Incorrect. Manually alternating requests is error-prone and inefficient compared to Google Cloud's built-in traffic-splitting feature.