Google Professional Machine Learning Engineer Question 368
Select 3Google Cloud PlatformYou are building a machine learning model to predict customer churn for a subscription-based business. After training the model, you observe unusually high accuracy during evaluation. To validate the model and ensure its reliability, which of the following steps should you take?
- A
Check for data leakage by analyzing the features used in the model.
- B
Use a different evaluation metric, such as precision, recall, or F1-score, to assess performance.
- C
Manually inspect a subset of the test dataset to ensure the labels are correct.
- D
Increase the size of the test set to improve statistical significance.
- E
Examine the distribution of the training and test datasets to ensure they are consistent.
Show answer and explanation
Correct answers: A, B, E
Explanation
Validating a machine learning model involves identifying potential issues such as data leakage, ensuring the evaluation metrics are appropriate for the problem, and confirming consistency between the training and test datasets. These steps help ensure that the model's performance is reflective of real-world scenarios and not due to artifact or bias in the data or evaluation process.
- A. Correct.
Data leakage can lead to artificially inflated model performance by incorporating information in the training data that would not be available during real-world predictions. This is critical to check during validation.
- B. Correct.
Accuracy alone may not be sufficient, especially for imbalanced datasets. Using other metrics like precision, recall, or F1-score provides a more comprehensive evaluation of the model's performance.
- C. Incorrect.
While label accuracy is important, manually inspecting a subset of the test dataset is not a standard or scalable validation approach for detecting issues like overfitting or data leakage.
- D. Incorrect.
Increasing the size of the test set may improve statistical significance but does not directly address validation issues such as overfitting, distribution mismatch, or data leakage.
- E. Correct.
Ensuring consistent data distribution between the training and test datasets is crucial for reliable model validation. A mismatch can lead to overly optimistic or pessimistic evaluation results.