:quality(75))
- The post walks through running a GLM model in H2O from R, based on notes from a September 26 hands on meetup.
- The post lists three prerequisites: a recent version of R, the H2O jar file, and the h2oWrapper R package that connects R to H2O.
- The post demonstrates starting H2O with the command java -Xmx3g -jar h2o.jar -name mystats-cloud1, allocating 3 gigabytes of memory.
- The post describes a demo run on the full airlines dataset of 152,360,031 observations using every available variable.
- The post warns that if the H2O instance running in the background stops, R loses its connection to the cluster and any unsaved work.
This is a walk through of running H2O through R. Before you get started you will need three things: R (a recent version), H2O (wich you can get through github: https://github.com/0xdata/h2o) or directly from our website: http://0xdata.com/h2O/, and the h2oWrapper R package, which is the tool that makes H2O talk to R, and lets you talk to big data. We’re going to run H2O from your computer. To do this open your command line terminal (or wherever you run java programming from). If you’re doing this on a server, you need the same basic elements, and permissions, but other than that, everything is just the same. Start an instance of H2O: CD to the directory with the h2o jar </blah file path to working directory>/h2o-1.7.0.536 Enter the java command:
In the command above where you see memory specify the amount of memory you want to allocate to h2o (if you’re not sure- 3 gigs is a good starting place): java -Xmx3g -jar h2o.jar -name mystats-cloud1. Ok. You have an instance of h2o running? Good. Now go to R. In the R console either change your working directory or be ready to give R an absolute path to the R package. This package makes sure that the version of h2o in R and the version of h2o you are running in your cloud are the same and can talk to each other.
This installs the dependent packages that you need to run h2o from R:
Make an R object for you h2o instance (the h2o cloud that you started from your terminal)
Starts H2O in R
Beginner calls in R for running a glm model in H2O: (you can get help and a full listing of functions and calls by calling ?? h2o)
generate predictions for a new set of data using a glm model you built in H2O.
Here is the code from a demo we ran at a hands on with the full airlines data set (when I say full I mean ALL observations, and all possible variables. 152,360,031 observations). That’s a huge amount of data, so this is probably a good time to talk about the difference between h2o in R and R. H2O in R is happy to call basic R functions, and you can use H2O with R to your heart’s content, but when you run an instance of h2o in R, you’re using H2O math, H2O objects and H2O distributed computing. R is acting as the user interface. This has a few implications. The first is that if you want to run really big data through R using H2O you need to do it on a server. Our compression scheme is amazing, but the full airlines data set won’t do well with only 4 gigs of memory. The second is that you need an instance of H2O running while you work in R. If your instance of H2O dies, or you stop it, H2O R no longer has cluster on the other side of the program to talk to, and you will loose your work. I really highly recommend saving your R work in an external script editor, so that you don’t have to recreate your R code from scratch if a server goes down.
I found the IP and Port by looking the output from the terminal where I started H2O.
The curious can also open the web browser, point it at the IP and port number specified in your java command output, and watch all of the work you are doing in H2O show up in your jobs list. You can also continue your analysis in the web browser. Have fun!