MLS-C01 Question 151
Select 3A data scientist at a retail company has built a machine learning model to predict customer churn using Amazon SageMaker. The stakeholders, however, are concerned about understanding why the model makes certain predictions. Which approach can the data scientist take to improve the stakeholders' understanding of the model's predictions?
- A
Use Amazon SageMaker Clarify to generate SHAP (SHapley Additive exPlanations) values for feature importance.
- B
Enable Amazon CloudWatch logs to track the model's decision-making process step by step.
- C
Train a simpler linear regression model as a baseline and compare its interpretability with the existing model.
- D
Use LIME (Local Interpretable Model-Agnostic Explanations) to explain specific predictions locally.
- E
Switch to a rule-based algorithm that inherently explains decisions, such as decision trees, instead of the current model.
Show answer and explanation
Correct answers: A, C, D
Explanation
To help stakeholders understand the intuition behind a machine learning model, techniques like SHAP and LIME provide insights into feature importance and specific predictions. Additionally, comparing the current model with a simpler one can illustrate the trade-offs between accuracy and interpretability. Monitoring tools like CloudWatch or switching to rule-based algorithms are not direct solutions to improving model explainability for stakeholders.
- A. Correct.
SHAP values are an effective way to explain the global and local feature importance of a model's predictions, making it easier for stakeholders to understand the intuition behind the model.
- B. Incorrect.
While CloudWatch logs are useful for monitoring and debugging, they do not provide explanations or insights into model predictions.
- C. Correct.
Training a simpler model like linear regression can provide a benchmark for interpretability and help stakeholders understand the trade-off between complexity and explainability.
- D. Correct.
LIME is a model-agnostic technique that explains individual predictions by approximating the model locally, which can help stakeholders understand specific cases.
- E. Incorrect.
Switching to a rule-based algorithm is not always feasible or desirable, as it may compromise the model's accuracy or fail to meet the use case requirements.