Interpretable Machine Learning Using LIME Framework - Kasia Kulma (PhD), Data Scientist, Aviva
- Predictive AI
- H2O Feature Store
Overview
In this talk, Kasia Kulma, a data scientist at Aviva, explains why black box machine learning models need interpretability tools and walks through the LIME framework (Local Interpretable Model-agnostic Explanations). She covers the accuracy versus interpretability trade-off, how LIME works under the hood by perturbing data around a single observation and fitting a simple local model to explain it, and demos it on a breast cancer classification model built with H2O AutoML. She also covers real-world stakes, including GDPR's right to an explanation and examples where models learned the wrong signal entirely.
Key takeaways
- Why black box models like neural nets and gradient boosted trees are hard to trust, and why tools like feature importance and partial dependency plots fall short, especially for explaining a single prediction
- How LIME actually works: it perturbs an observation's data to build a local fake dataset, scores similarity to the original, runs the black box model's predictions on it, then fits a simple local linear model to produce explanation weights
- The husky versus wolf example, where a model was 94% accurate but had actually learned to detect snow in the background rather than the animal, a failure interpretability caught that accuracy alone would have missed
- A live demo applying LIME to a breast cancer classification model trained with H2O AutoML, showing which cell features (size, shape, mitosis count) drove correct predictions and which drove misclassifications
- Why interpretability matters for trust, for debugging (letting non-experts improve models through feature engineering), and for compliance, including GDPR's requirement that customers receive an explanation for automated decisions
Start owning your AI