Create comprehensive, high-quality model documentation in minutes that saves time, increases productivity and improves model governance.

The Model Documentation Paradox

For many companies, model documentation is a requirement for any model to be used in the business. For other companies, model documentation is part of a data science team’s best practices. Model documentation includes how a model was created, training and test data characteristics, what alternatives were considered, how the model was evaluated, and information on model performance. Collecting and documenting this information can take a data scientist days to complete for each model. The model document needs to be comprehensive and consistent across various projects. The process of creating this documentation is tedious for the data scientist and wasteful for the business because the data scientist could be using that time to build additional models and create more value. Inconsistent or inaccurate model documentation can be an issue for model validation, governance, and regulatory compliance.

H2O AutoDoc 

H2O Automated Model Documentation (AutoDoc) automatically creates model documentation for supervised learning models created in H2O-3 and Scikit-Learn. Automated documentation has been used in production in H2o Driverless AI. This industry-leading capability is now available to everyone who uses H2O-3.

Key Capabilities 

• Automatic document generation in Microsoft Word (docx) or Markup (.md) formats.
  
  
• Out-of-the-box documentation template included
  
  
• Template customization available to fit with your organization’s standards and requirements
  
  
• Support for models generated in H2o-3 and ScikitLearn
  
  
• Support for H2O-3: Deep Learning, Distributed Random Forrest, GLM, Gradient Boosted Machines, Stacked Ensembles, and XGBoost models.

Documentation Features 

The Word editable document generated has these standard automated documentation template and includes:

• Experiment Overview to provide an overview of the modeling problem
  
  
• System Specifications to describe the exact configuration of the system that produced the model including the version of H2o-3 or Scikitlearn that was used.
  
  
• Data Overview including information on the data shape and summary statistics for each feature (numeric and categorical values)
  
  
• Data Shift to highlight any difference between training and validation data.
  
  
• Validation Strategy
  
  
• Model Parameters and Values
  
  
• Common Classification or Regression Metrics
  
  
• Population Stability Index
  
  
• Prediction Statistics for training and validation datasets
  
  
• Feature Importance using H2O native importance or Shapely Importance
  
  
• Response Rate by Quantile
  
  
• Actual vs. Predicted Probabilities
  
  
• Partial Dependence Plots
  
  
• Alternative Models Summary to show the other techniques and their parameters that were tested against the winning model