Disparate Impact Analysis for Model Fairness
- Predictive AI
- H2O Driverless AI
This lesson explains Driverless AI's disparate impact analysis, a way to evaluate whether a classification model treats different groups fairly. Using a loan approval model as the example, it shows how to pick a variable like gender, choose a reference level such as female to compare against male, and read the resulting proportion to judge whether outcomes are distributed fairly across groups. It also covers reading the metrics section, including the F1 score, the cutoff probability, and the confusion matrix, and points to custom fairness and bias recipes available in H2O's public GitHub repository.
Key takeaways
- How disparate impact analysis compares the proportion of favorable outcomes between a reference group and another group
- What an adverse impact ratio above the four-fifths threshold suggests about a model's fairness toward the groups being compared
- How to read the metrics section: the F1 score as the maximized metric, the cutoff probability, and the confusion matrix by group
- How to find publicly available custom recipes on H2O's GitHub repository for extending Driverless AI's fairness checks
Start owning your AI