August 5th, 2022

Make with Recap: Getting Started with H2O Document AI

RSS icon RSS Category: Deep Learning, H2O Document AI, Make with, NLP

Product Owner, Data Scientist, and Kaggle Grandmaster, Mark Landry presented at the Make with session on getting started with H2O Document AI. 

The session covered an overview of H2O Document AI, a tool to extract insights and automate document processing. The session also included a product demo, looking at documents as data sets, how to do annotation with the tool, how to create targets, modeling and feedback, and publishing a pipeline. 

H2O Document AI Overview

H2O Document AI can work with all types of documents such as PDFs, pictures of documents, faxes, and text documents. The process consists of pre-processing that is built into the tool, optical character recognition (OCR) models, labeling, training models (incorporating multiple models into the same pipeline), and post processing, taking the results of those models and deploying them. 

The tool is not just for data scientists and annotators but as Landry puts simply, “the intent of our tool is to be simple enough to … train other people familiar with data, but not necessarily data scientists to be able to operate this tool as well.” The real value in H2O Document AI is that the tool is completing document processing tasks that humans are doing today. Beyond this, there is a REST API for deployment, which allows flexibility for how the data is consumed, whether that be in a data store, building a user interface on top of it, or integration with business applications and other tools. Lastly, there is a human in the loop: review/ correction aspect built into the tool “so you can stand up a model, and then work it better and better and better as you collect more documents … we can annotate from scratch, or we can annotate from predictions.”

Example: Medical Referral 

The image above is an example document of a physician referral from a use case with UCSF. Physician referral documents are sent from one practice to another and there is no control over the format, it is not templatized. As such is the case for most customers across various industries who have a situation where they receive thousands of different document formats, but this tool can handle the various formats. 

There are likely multiple classes the customer is interested in pulling from these documents (116 in the case with UCSF). H2O Document AI uses deep learning models that can handle 116 classes of information with the same efficiency as 5 classes. The tool can pull information from various types of documents. See another document example below:

Model Generalizes Other Formats

The format here is totally different from the referral form above, though each of these forms contain similar content. 

The tool uses advanced deep learning models, transformer models, and high end NLP models (very similar to BERT). These documents aren’t read in typical reading order, reading left to right. The models are understanding the structure of the documents as a whole and table structures featured in the documents. Using transfer learning, the variants are pretrained models that have seen 11 million documents previously. The tool can process different document types, even documents that are more difficult to read can be managed with the OCR, shown in the examples below: 

Any Document Type

H2O Document AI is saving customers thousands of hours compared with having humans complete these tasks.  

Watch the product demo and recap of the Make with session: Getting Started with H2O Document AI. Attend an upcoming Make with session

About the Author

Blair Averett

Blair Averett is the Head of Digital Media on the Marketing team at Blair manages content marketing, paid media and social media.

Leave a Reply

[Infographic] Healthcare providers: How to avoid AI “Pilot-Itis”

From increased clinician burnout and financial instability to delays in elective and preventative care, the

March 15, 2023 - by
Deploy a WAVE app on an AWS EC2 instance

This article was originally published by Greg Fousas and Michelle Tanco on Medium  and reviewed by

March 10, 2023 - by Michelle Tanco and Greg Fousas
How Horse Racing Predictions with Saved a Local Insurance Company $8M a Year

In this Technical Track session at H2O World Sydney 2022, SimplyAI's Chief Data Scientist Matthew

March 8, 2023 - by Liz Pratusevich
AI and Humans Combating Extinction Together with Dr. Tanya Berger-Wolf

Dr. Tanya Berger-Wolf, Co-Founder and Director of AI for conservation nonprofit Wild Me, takes the

March 1, 2023 - by Liz Pratusevich
Improving Search Query Accuracy: A Beginner’s Guide to Text Regression with H2O Hydrogen Torch

Although search engines are vital to our daily lives, they need help understanding complex user

February 28, 2023 - by
Blog header image with boats on it
What it means—and takes—to be at AI’s edge with Dr. Tim Fountaine

Dr. Tim Fountaine, Senior Partner at McKinsey & Company joins us at H2O World Sydney

February 23, 2023 - by Liz Pratusevich

Request a Demo

Explore how to Make, Operate and Innovate with the H2O AI Cloud today

Learn More