March 24th, 2014

Data Munging in H2O+R

RSS icon RSS Category: Uncategorized [EN]
Fallback Featured Image

Over the weekend we fielded a question from one of our users about the basics of data munging in H2O through R – and it was a good question, so I wanted to share the response with a wider audience – namely you guys.

There are a few quick things about data munging in  H2O+R:
– It often looks and feels like you are manipulating data in R; we designed it to work that way.  However, for all of the trappings of old R, once you've passed data to H2O, all of your data munging is taking place on the H2O cluster, and the information you see is being passed to R through JSON.
– This means that you aren't limited by the ceiling on R's ability to handle data, you are now limited by the total amount of memory you initially allocated to your cluster.
– It also means that some commands should be undertaken with care. For example, it's now possible to manipulate datasets with thousands of factor levels. Asking H2O+R to return a table displaying information from high cardinality factors can produce results of overwhelming volume.
-If it's necessary to pass data back and forth from H2O to R (i.e., if you want to manipulate data in the R environment, and not on the H2O server) the R calls str(), and as.h2o() are your new best friends*.
In this context data set) turns data into a data frame in the R environment.  Use it wisely – it's entirely possible to ask H2O to pass R many millions of observations and quickly exceed R's capacities – turning your R session into a paperweight. I highly recommend that if you must take data from H2O into R, that you take only the data you absolutely need- pass in a single column or two, but avoid moving the whole large data set if you can.
On the other hand, as.h2o(R data set) can take data from R and pass it to the server. Here the limitation is the amount of memory allocated to H2O, and frankly, R will top out long before H2O does – so if you can work with data in R, you can surely pass it to your cluster.
Double checking that your data were communicated correctly uses str(data set). It's a good double check when you've passed information back and forth to ensure that factors are still being treated as factors, that NA's have been treated appropriately, and all of the other little details that are important moving forward with analysis.
*There are examples in the attached R script, as well as the data set for those examples.  It's easy enough to cut and paste or command+enter through the R file, but you will need to 1. Start your own instance of h2o (or at least have one available) before getting to work and 2.Specify your own file path in line 7 – the path that is included is a place holder, and it will only match your path by accident.
Get the Data-
Get the R script –

Leave a Reply

H2O LLM DataStudio Part II: Convert Documents to QA Pairs for fine tuning of LLMs

Convert unstructured datasets to Question-answer pairs required for LLM fine-tuning and other downstream tasks with

September 22, 2023 - by Genevieve Richards, Tarique Hussain and Shivam Bansal
Building a Fraud Detection Model with H2O AI Cloud

In a previous article[1], we discussed how machine learning could be harnessed to mitigate fraud.

July 28, 2023 - by Asghar Ghorbani
A Look at the UniformRobust Method for Histogram Type

Tree-based algorithms, especially Gradient Boosting Machines (GBM's), are one of the most popular algorithms used.

July 25, 2023 - by Hannah Tillman and Megan Kurka
H2O LLM EvalGPT: A Comprehensive Tool for Evaluating Large Language Models

In an era where Large Language Models (LLMs) are rapidly gaining traction for diverse applications,

July 19, 2023 - by Srinivas Neppalli, Abhay Singhal and Michal Malohlava
Testing Large Language Model (LLM) Vulnerabilities Using Adversarial Attacks

Adversarial analysis seeks to explain a machine learning model by understanding locally what changes need

July 19, 2023 - by Kim Montgomery, Pramit Choudhary and Michal Malohlava
Reducing False Positives in Financial Transactions with AutoML

In an increasingly digital world, combating financial fraud is a high-stakes game. However, the systems

July 14, 2023 - by Asghar Ghorbani

Ready to see the platform in action?

Make data and AI deliver meaningful and significant value to your organization with our state-of-the-art AI platform.