Google Professional Machine Learning Engineer Question 483
Select 3Google Cloud PlatformYou are managing a deployed machine learning model on Google Cloud that recommends products to users. Recently, you observed a sudden drop in the model's performance, as customer engagement metrics have significantly decreased. Which of the following steps should you take to monitor, test, and troubleshoot the model’s issues?
- A
Analyze the distribution of incoming feature data to check for data drift compared to the training dataset.
- B
Inspect the model's hyperparameter values and immediately re-tune them to improve performance.
- C
Set up logging on prediction requests and responses to identify patterns or anomalies in the inputs and outputs.
- D
Check for changes in the upstream data pipeline to see if there are issues with data preprocessing.
- E
Deploy a completely new model trained on the same dataset without investigating the root cause of the issue.
Show answer and explanation
Correct answers: A, C, D
Explanation
To troubleshoot the drop in model performance, you need to systematically monitor and analyze potential causes such as data drift, anomalies in inputs and outputs, and issues in the data pipeline. These steps allow you to pinpoint the root cause before deciding on corrective actions like retraining or re-tuning. Skipping diagnostics or deploying a new model prematurely can exacerbate the problem and waste resources.
- A. Correct.
Analyzing the distribution of incoming feature data is critical for identifying data drift, which is a common reason for performance degradation in deployed models.
- B. Incorrect.
While hyperparameter tuning can be useful, immediately re-tuning them without investigating the root cause is not a recommended first step in troubleshooting.
- C. Correct.
Setting up logging on prediction requests and responses helps identify anomalies, edge cases, or patterns in the data that can lead to performance issues.
- D. Correct.
Checking upstream data pipelines is crucial, as changes or errors in data preprocessing can result in input data that is inconsistent with the model's expectations.
- E. Incorrect.
Deploying a new model without identifying the root cause of the issue is not a best practice and may lead to the same problem recurring.