Scott Lundberg, Microsoft Research - Explainable Machine Learning with Shapley Values - #H2OWorld
- Observability
- H2O MLOps
Overview
In this H2O World talk, Microsoft Research's Scott Lundberg explains SHAP (SHapley Additive exPlanations), a method for attributing a model's prediction to its individual input features using game-theoretic Shapley values. Using a loan-risk example, he shows how feature attributions can reveal an accurate model that has learned something wrong, and extends the technique to model monitoring by decomposing prediction loss over time to catch errors after deployment.
Key takeaways
- Why explaining a single prediction is more tractable than explaining an entire complex model, and why that matters for high-stakes decisions like loan approvals
- How Shapley values fairly attribute a prediction's difference from the baseline across each input feature, satisfying efficiency and consistency guarantees
- Why feature order matters for simple attribution approaches, and how Shapley values resolve that using game theory dating back to the 1950s
- How the SHAP package computes efficient, exact attributions for tree-based models like random forests and gradient boosted trees
- How a real credit risk model was found, through SHAP plots, to be flagging retirement-age applicants based on long credit history, a signal of a debugging problem
- How decomposing prediction loss, rather than just output, by feature over time supports model monitoring, catching an introduced bug in a hospital procedure-duration model that a plain loss chart missed
Start owning your AI