Sparkling Water 2.3.0 is now available!

Key takeaways

Sparkling Water 2.3.0 added full support for Spark 2.3 while keeping the Spark 2.2 and 2.1 releases updated with the latest fixes.
Sparkling Water 2.3.0 fixed a bug so H2OMojoModel now handles null values as missing data instead of failing predictions.
Sparkling Water 2.3.0 fixed PySparkling's as_h2o_frame method so it now raises an error on an unsupported input type instead of silently doing nothing.
Sparkling Water 2.3.0 dropped Scala 2.10 support for Spark 2.3, matching Spark 2.3's own removal of Scala 2.10, though older Spark versions still support it.
Sparkling Water 2.3.0 integrated with H2O 3.18.0.5, bringing in that release's fixes alongside Sparkling Water's own changes.

Hi Makers! We are happy to announce that Sparkling Water now fully supports Spark 2.3 and is available from our download page . If you are using an older version of Spark, that’s no problem. Even though we suggest upgrading to the latest version possible, we keep the Sparkling Water releases for Spark 2.2 and 2.1 up-to-date with the latest version if we are not limited by Spark. The last release of Sparkling Water contained several important bug fixes. The 3 major bug fixes are:

Handle nulls properly in H2OMojoModel. In the previous versions, running predictions on the H2OMojoModel with null values would fail. We now handle the null values as missing values and it no longer fails.
We marked the Spark dependencies in our maven packages as provided. This means that we assume that Spark dependencies are always provided by the run-time, which should always be true. This ensures a cleaner and more transparent Sparkling Water environment.
In PySparkling, the method as_h2o_frame didn’t issue an alert when we passed in a wrong input type. This method accepts only Spark DataFrames and RDDs, however, some users tried to pass different types and this method ended silently. Now we fail if the user passes a wrong data type to this method.

It is also important to mention that Spark 2.3 removed support for Scala 2.10. We’ve done the same in the release for Spark 2.3. Scala 2.10 is still supported in the older Spark versions. The latest Sparkling Water versions also integrated with H2O 3.18.0.5 which brings several important fixes. The full change log for H2O 3.18.0.5 is available here and the full Sparkling Water change log can be viewed here . Thank you! Kuba Senior Software Engineer, Sparkling Water Team

Key takeaways

Sparkling Water 2.3.0 added full support for Spark 2.3 while keeping the Spark 2.2 and 2.1 releases updated with the latest fixes.
Sparkling Water 2.3.0 fixed a bug so H2OMojoModel now handles null values as missing data instead of failing predictions.
Sparkling Water 2.3.0 fixed PySparkling's as_h2o_frame method so it now raises an error on an unsupported input type instead of silently doing nothing.
Sparkling Water 2.3.0 dropped Scala 2.10 support for Spark 2.3, matching Spark 2.3's own removal of Scala 2.10, though older Spark versions still support it.
Sparkling Water 2.3.0 integrated with H2O 3.18.0.5, bringing in that release's fixes alongside Sparkling Water's own changes.

Handle nulls properly in H2OMojoModel. In the previous versions, running predictions on the H2OMojoModel with null values would fail. We now handle the null values as missing values and it no longer fails.
We marked the Spark dependencies in our maven packages as provided. This means that we assume that Spark dependencies are always provided by the run-time, which should always be true. This ensures a cleaner and more transparent Sparkling Water environment.
In PySparkling, the method as_h2o_frame didn’t issue an alert when we passed in a wrong input type. This method accepts only Spark DataFrames and RDDs, however, some users tried to pass different types and this method ended silently. Now we fail if the user passes a wrong data type to this method.