MLS-C01 Question 377
Select 3You have deployed a machine learning model for predicting customer churn using Amazon SageMaker. However, during testing, you notice that the model consistently predicts high churn probabilities for all inputs, regardless of customer attributes. Which of the following steps would most likely help debug and resolve this issue?
- A
Check the distribution of the training dataset to ensure it is balanced.
- B
Inspect the feature importance to verify that the model is using relevant features.
- C
Enable Amazon SageMaker Debugger to identify anomalies during training.
- D
Increase the size of the training dataset by adding more samples.
- E
Use Amazon SageMaker Model Monitor to check for data drift in the input features.
Show answer and explanation
Correct answers: A, B, C
Explanation
The issue of consistently high churn predictions suggests that there might be a problem with the training data, feature selection, or the training process itself. Checking for imbalanced data, inspecting feature importance, and using SageMaker Debugger to identify training anomalies are targeted approaches to debug this issue. Increasing the dataset size or using Model Monitor addresses different concerns and are not directly relevant to this scenario.
- A. Correct.
If the training dataset is imbalanced (e.g., if one class dominates), the model may become biased and predict the same class frequently. Checking the dataset's class distribution is a critical step in debugging.
- B. Correct.
Inspecting feature importance can reveal if the model is relying on irrelevant or redundant features, which could lead to poor performance.
- C. Correct.
Amazon SageMaker Debugger can help identify issues during training, such as gradient anomalies or other misconfigurations, which may explain the model's incorrect behavior.
- D. Incorrect.
Although increasing the size of the training dataset can improve model performance, it is not a targeted debugging step for this specific issue. The root cause should be identified first.
- E. Incorrect.
Amazon SageMaker Model Monitor is useful for detecting data drift in deployed models, but in this case, the issue is related to the training process, not post-deployment data drift.