New Improvements in H2O 3.32.0.2

Key takeaways

H2O 3.32.0.2 added feature interaction constraints to its XGBoost integration, letting users decide which variables were allowed to interact in a model.
H2O 3.32.0.2 introduced an XGBFI-style tool that ranked feature interactions using gain, cover, and frequency measures, plus leaf statistics and split value histograms.
Gain measured a feature's relative contribution to the model's predictions, while cover measured the number of observations affected by a feature's splits.
H2O published a Jupyter notebook demo showing how to train an XGBoostEstimator with interaction constraints and display the resulting feature interactions.

There is a new minor release of H2O that introduces two useful improvements to our XGBoost integration: interaction constraints and feature interactions.

Interaction Constraints

Feature interaction constraints allow users to decide which variables are allowed to interact and which are not.

Potential benefits:

Better predictive performance from focusing on interactions that work – whether through domain-specific knowledge or algorithms that rank interactions
Less noise in predictions; better generalization
More control given to the user on what the model can fit. For example, the user may want to exclude some interactions even if they perform well due to regulatory constraints

(Source: https://xgboost.readthedocs.io/en/latest/tutorials/feature_interaction_constraint.html )

The H2O documentation is available here .

XGBFI-like Tool for Revealing Feature Interactions

We have implemented ranks of features and feature interactions by various measures in XGBFI style. Thanks to this tool, H2O provides insights into higher-order interactions between features in trees all in a user-friendly manner. Additionally, leaf statistics and split value histograms are provided. The measures used are either one of:

Gain implies the relative contribution of the corresponding feature to the model calculated by taking each feature’s contribution for each tree in the model. A higher value of this metric when compared to another feature implies it is more important for generating a prediction.

Cover is a metric to measure the number of observations affected by the split. Counted over the specific feature it measures the relative quantity of observations concerned by a feature.

Frequency (FScore) is the number of times a feature is used in all generated trees. Please note that it does not take the tree-depth nor tree-index of splits a feature occurs into consideration, neither the amount of possible splits of a feature. Hence, it is often suboptimal measure for importance or their averaged / weighed / ranked alternatives.

The H2O documentation is available here .

Example

The Jupyter notebook demo with all example codes presented below is available here .

Train XGBoostEstimator with interaction_constraints parameter:

The result:

Display feature interactions:

Credits

This new H2O release is brought to you by Veronika Maurerova , Zuzana Olajcova , and Hannah Tillman .

How to Get Started?

Download H2O-3 from here and follow the steps in this example notebook . You can also check out our training center for both self-paced tutorials and instructor-led courses.

Key takeaways

H2O 3.32.0.2 added feature interaction constraints to its XGBoost integration, letting users decide which variables were allowed to interact in a model.
H2O 3.32.0.2 introduced an XGBFI-style tool that ranked feature interactions using gain, cover, and frequency measures, plus leaf statistics and split value histograms.
Gain measured a feature's relative contribution to the model's predictions, while cover measured the number of observations affected by a feature's splits.
H2O published a Jupyter notebook demo showing how to train an XGBoostEstimator with interaction constraints and display the resulting feature interactions.

There is a new minor release of H2O that introduces two useful improvements to our XGBoost integration: interaction constraints and feature interactions.

Interaction Constraints

Feature interaction constraints allow users to decide which variables are allowed to interact and which are not.

Potential benefits:

Better predictive performance from focusing on interactions that work – whether through domain-specific knowledge or algorithms that rank interactions
Less noise in predictions; better generalization
More control given to the user on what the model can fit. For example, the user may want to exclude some interactions even if they perform well due to regulatory constraints

(Source: https://xgboost.readthedocs.io/en/latest/tutorials/feature_interaction_constraint.html )

The H2O documentation is available here .

XGBFI-like Tool for Revealing Feature Interactions

Cover is a metric to measure the number of observations affected by the split. Counted over the specific feature it measures the relative quantity of observations concerned by a feature.

The H2O documentation is available here .

Example

The Jupyter notebook demo with all example codes presented below is available here .

Train XGBoostEstimator with interaction_constraints parameter:

The result:

Display feature interactions:

Credits

This new H2O release is brought to you by Veronika Maurerova , Zuzana Olajcova , and Hannah Tillman .

How to Get Started?

Download H2O-3 from here and follow the steps in this example notebook . You can also check out our training center for both self-paced tutorials and instructor-led courses.