:quality(75))
- The post shared a Scala code sample, AirlinesDemo, that queried an airlines dataset in Spark using a SQL-style filter for flights destined to SFO.
- The code connected to a Shark cluster to extract data, transferred it into H2O, and then called a GBM model directly through H2O's Java API.
- The sample model in the code predicted the isDepDelayed column using 100 trees in H2O's Gradient Boosting Machine.
object AirlinesDemo extends Demo { override def run(conf: DemoConf): Unit = { // Prepare data // Dataset val dataset = “data/allyears2k_headers.csv” // Row parser val rowParser = AirlinesParser // Table name for SQL val tableName = “airlines_table” // Select all flights with destination == SFO val query = “””SELECT * FROM airlines_table WHERE dest=”SFO” “”” // Connect to shark cluster and make a query over prostate, transfer data into H2O val frame:Frame = executeSpark<a href="dataset, rowParser, conf.extractor, tableName, query, local=conf.local">Airlines</a> Log.info(“Extracted frame from Spark: “) Log.info(if (frame!=null) frame.toString + “\nRows: “ + frame.numRows() else “<nothing>“)</nothing> // Now make a blocking call of GBM directly via Java API val model = gbm(frame, frame.vec(“isDepDelayed”), 100, true) Log.info(“Model built!”) } override def name: String = “airlines” }