Last updated: 09/23/19 

H2O Driverless AI provides a great new feature called “custom recipes”. These recipes are essentially custom snippets of code which can incorporate any machine learning algorithm , any scorer/metric and any feature transformer. A user can create custom recipes using python utilizing any external library or his/her own creations. This is a interesting feature because it allows Driverless AI’s automated approach to be enhanced by the data scientist’s expert domain knowledge on his/her field. If you’d like to understand more about DAI’s custom recipes, you may have read about it on H2O.ai’s website  and/or look specifically in the appendix section .

As part of this, we have created a github repository  where we share some of our own custom recipes . There are different folders which cover different ML areas, for example NLP, Time Series, categorical features, numerical features, geospatial and so on.

The API to create such snippets is similar to sklearn’s . There are `fit()` and `transform()` methods for transformers, `fit()` and `predict()` methods for the models as well as a `score()` method for metrics. Anybody can contribute to that repository and help others utilize more tools/methods in any domain.

Two of the most popular recipes in that repo are the Prophet and pmdarima (for ARIMA) implementations which can be integrated with our time series  forecasting recipe. They appear under the folders `driverlessai-recipes/transformers/timeseries/ ` with the names parallel_auto_arima_forecast.py  and parallel_prophet_forecast.py. 

A small note: If interested to know more about the Times series functionality within Driverless AI, you could read about it from H2O.ai’s documentation. 

Back to the recipes. From inside DAI’s EXPERIMENT page, you can upload these scripts using the “+ UPLOAD CUSTOM RECIPE” button as shown below.

After doing some consistency checks to validate that no errors occur when different datasets are fit using these recipes, Driverless AI will integrate these to the list of transformers/models/metrics available within the tool.

Then from the RECIPIES’ tab, the user could select or deselect transformers (or models or metrics):

These two are technically machine learning models; however, we use them as transformers – you may think of them as a much more advanced versions of moving averages. At any point in time and for each one of the time groups (as selected from within Driverless) a new Autoarima or Prophet model is fit using historical data to predict the current point in time.

For example. We want to create the Prophet feature that estimates sales for month 4 for store 1 and department 2 (e.g the time groups). A prophet model will be trained using all the available past (3) observations (months 1,2,3) and predictions will be made for month 4 – it is quite similar as if you were estimating a moving average using the historical sales from the same time stamps. We do this for all time points and all combinations of groups. Then we use other machine learning models (such as lightgbm or xgboost) to find the best way to combine the Prophet’s (or Autoarima’s) predictions along with any other time series features we have created for the same time.

From experience, these features tend to give good uplift on top of Driverless AI’s native time series features. Here is why:

Prophet

Prophet was created by Facebook Research. This is the reference of the official paper: Taylor, S.J. and Letham, B., 2018. Forecasting at scale. The American Statistician , 72 (1), pp.37-45.

Link to the github repo: https://github.com/facebook/prophet 

What is it? According to the official repo: Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works best with time series that have strong seasonal effects and several seasons of historical data. Prophet is robust to missing data and shifts in the trend, and typically handles outliers well 

Prophet is described as a “curve-fitting” exercise where the main independent feature is time. This means Prophet does not use for example the concepts of “lags” to make predictions but tries to find relationships of how the target variable  will “move” based on describing its past relationship with the time variable. This can yield several advantages – my favourite of which – is that it can handle missing values in the times series.

To be more precise, the Prophet model is described from the bellow generic formula:

The formula connotes that the prediction of Prophet is consisted of 3 different components (or 3 different modelling approaches if you prefer). Prophet is already “an ensemble” (and in fact weighted to account for various elements) of models which makes it quite powerful compared to other individual/single features.

A nice summary of prophet is explained in  this article . Information from that article is also borrowed in this post.

g(t) is a Piecewise linear or logistic (growth) regression to calculate trend 

Regression  models that also account for saturation growth are fit to capture trend of the series. In addition to that, Prophet introduces the concept of “changepoints” in trends. At different timestamps (which can also be provided manually by the user), it will fit different regressions in order to capture shifts in trends.

For instance, if we had the following series:

Prophet could first fit a regression:

Then in it can use a later changepoint of trend and fit another line in order to reduce the overall error. For example, it may look like:

These examples (e.g. the lines) are made up, just to show you generically how Prophet could use the changepoints of trend to refit models and describe the overall time series (trends-wise) better.

If this was the time series:

The trends’ component would be:

The second component:

s(t) models periodic changes (e.g. weekly/yearly seasonality)

It is mapped with the following formula:

This is a Fourier series, where parameters a and b are unknown (and Prophet tries to “tune/find” them) in order to best describe the periodicity/seasonality of the series based on different frequencies. It will run multiple such series to best capture the peaks and bottoms that may happen within a year, within a month and/or within a week and other frequencies.

For example, assuming we have the same above series, the yearly Seasonal component is described by:

The monthly component:

And the weekly component:

All these models are combined though another model to create forecasts.

Then there is also the:

h(t) holiday  component. Through this you can provide a separate file, highlighting which days are deemed as holidays, so Prophet can learn to take them into account through another model. E.g the holiday component could look like:

To sum up, the prophet model is an ensemble of 3 modelling approaches:

• Capturing trends
• Capturing periodicity of different frequencies
• Capturing holiday aspects

As such (ensemble) and as a standalone feature, it often appears to be quite strong relative to the other individual time series features created by Driverless AI and very frequently provides a good boost in performance.

Autoarima

Similar with Prophet, Autoarima within the Driverless AI custom recipes comes from a python package that utilizes the well-known ARIMA  technique for time series modelling , adding some automated elements to it in order to fine-tune some of its parameters in order to achieve higher accuracy.

The package used in that recipeis pmdarima .

Breaking down the elements of ARIMA (reference from: https://machinelearningmastery.com/arima-for-time-series-forecasting-with-python/ _):

• AR: A model that uses the dependent relationship between an observation and some number of lagged observations.
• I: The use of differencing of raw observations (e.g. subtracting an observation from an observation at the previous time step) in order to make the time series stationary.
• MA:Moving Average. A model that uses the dependency between an observation and a residual error from a moving average model applied to lagged observations.

In other words, ARIMA is a linear model that fits on top of some special features derived from a time series. These features are in simple terms:

• Lags (e.g. historical target values)
• Transforming the target variable to be target minus a lagand fit the model with this transformed target (instead of the original one)
• Features which are based on errors produced from estimates made using moving averages of the lagged predictions.

All these elements can be tuned for best performance. For example, how many lags should we use? How far back should we go? 2 time stamps, 3, more? Same for the moving average features, how far should we go? Also, should we transform the target?

The Autoarima package does various experiments (though grid search) to find the best parameters for these ARIMA components trying to give the best possible individual estimate for each combination of time groups within the data (as implemented within Driverless AI).

Generally, the ARIMA model is quite solid for capturing trend and works better with non-stationary data (e.g. series that tend to keep going up or down rather than staying in the same place over long periods of time).  Hence when a dataset exhibits such properties, the ARIMA-based features provide a nice boost to the Driverless AI model and complement nicely its time series features (like weighted moving averages, lags, date-derived, exp. smoothing etc).

As additional reference, the following article, explains the technique quite well with sample code too (although it does not use the automated approach) : https://machinelearningmastery.com/arima-for-time-series-forecasting-with-python/ 

Conclusion

Providing custom recipes to Driverless AI is a great way to insert domain expertise to the experiments and push the accuracy of ml models even higher.  Many such recipes are available through H2O.ai’s github repo  which is open for anyone to use and people can contribute too. Two popular recipes for time series that use Prophet and pmdarima’s autoarima are mentioned in this article where it is explained how to use them as well as why they provide very useful additions to what is already built within Driverless AI.