:quality(75))
Key takeaways
- The post announces Sparkling Water 2.0.3, built on H2O core 3.10.1.2, adding the ability to connect to an existing H2O cluster from Sparkling Water.
- The post explains that this architectural change makes Sparkling Water less affected by Spark terminating its executors, improving stability with many H2O and Spark nodes.
- The post lists a series of bug fixes in the release, including corrections to PySparkling numeric handling, RDD to H2OFrame conversion tests, and SparkDataFrame key randomization.
This release has H2O core – 3.10.1.2
Important Feature:
This architectural change allows to connect to existing h2o cluster from sparkling water. This has a benefit that we are no longer affected by Spark killing it’s executors thus we should have more stable solution in environment with lots of h2o/spark node. We are working on article on how to use this very important feature in Sparkling Water 2.0.3. Release notes: https://0xdata.atlassian.net/secure/ReleaseNote.jspa?projectId=12000&version=16601
2.0.3 (2017-01-04)
- Bug
SW-152 – ClassNotFound with spark-submit
- SW-266 – H2OContext shouldn’t be Serializable
- SW-276 – ClassLoading issue when running code using SparkSubmit
- SW-281 – Update sparkling water tests so they use correct frame locking
- SW-283 – Set spark.sql.warehouse.dir explicitly in tests because of SPARK-17810
- SW-284 – Fix CraigsListJobTitlesApp to use local file instead of trying to get one from hdfs
- SW-285 – Disable timeline service also in python integration tests
- SW-286 – Add missing test in pysparkling for conversion RDD[Double] -> H2OFrame
- SW-287 – Fix bug in SparkDataFrame converter where key wasn’t random if not specified
- SW-288 – Improve performance of Dataset tests and call super.afterAll
- SW-289 – Fix PySparkling numeric handling during conversions
- SW-290 – Fixes and improvements of task used to extended h2o jars by sparkling-water classes
- SW-292 – Fix ScalaCodeHandlerTestSuite
- SW-178 – Allow external h2o cluster to act as h2o backend in Sparkling Water
- SW-282 – Integrate SW with H2O 3.10.1.2 ( Support for external cluster )
- SW-291 – Use absolute value for random number in sparkling-water in internal backend
- SW-295 – H2OConf should be parameterized by SparkConf and not by SparkContext
Please visit https://community.h2o.ai to learn more about it, provide feedback and ask for assistance as needed. @avkashchauhan | @h2oai