TimeH2O: Compact Foundation Models for Time Series Forecasting

We are excited to announce TimeH2O-v1, the largest model in TimeH2O, our family of foundation models for time series forecasting. On fev-bench, a benchmark of 100 real-world forecasting tasks, it ranks first of 32 models. On GIFT-Eval, it has the lowest error among zero-shot models. Three main ingredients power this performance: high-quality data, an optimized training pipeline, and a purpose-built architecture. For data, we combine real-world time series with synthetic data generated via temporal causal models. For efficiency, our training pipeline grades far more values in every batch. Finally, our flip-equivariant design guarantees that a flipped series gets the flipped forecast.

Fig. 1: Official leaderboards, as of 24 October 2026. fev-bench: top 5 models, 100 tasks, non-commercial models included. GIFT-Eval: zero-shot entries, ranked on aggregate MASE and CRPS.

Results

On fev-bench, TimeH2O-v1 has the top skill score (50.8) and win rate (91.9%) across 100 tasks, with non-commercial models included. On GIFT-Eval, it has the lowest point error (MASE 0.654) and probabilistic error (CRPS 0.441) among zero-shot models.

TimeH2O-v1-tiny, released on 13 October, is a 5.8M-parameter univariate model for when compute is tight. On GIFT-Eval, it reaches a MASE of 0.687 and a CRPS of 0.468. It uses a single patch size and has no covariate adapter; everything else below applies to both models.

Model

In our experiments, a decoder-only transformer worked best. On top of it, we added several tricks that improve training and inference efficiency, which we detail below.

Fig 2: (a) Model overview. (b) Packed training batch, hidden patches in yellow. (c) TimeH2O-v1 decoder blocks; group attention every fourth block.

Full size

Flip Equivariance

The first trick is flip equivariance. It is motivated by the idea that time series have no preferred direction: a spike is a dip flipped upside down, and a rising trend is a falling one flipped. A standard transformer has to learn the flipped version of every pattern during training. We build the symmetry into the architecture instead, so that when the input is flipped, TimeH2O returns the same forecast, flipped upside down.

Variable Patch Sizes

The second trick is variable patch sizes. Typically, time series foundation models (TSFMs) use fixed patches to represent chunks of a series as single tokens. While large patches improve efficiency, they lose fine details. A fixed size of 16 or 32 is a standard compromise, but variable patches go a step further.

During training, we randomly select a patch size (2, 4, 8, or 16) for each series among those that fit a training row. The architecture is shared, and the fixed row length keeps training efficient.

At inference, we use all patch sizes that fit a 1024-token window. This provides two more benefits: shorter series get more tokens to capture fine details, and the varying sizes act as a form of test-time augmentation.

Covariate Adapter

The third trick is the covariate adapter. Covariates are series that help predict the target, like temperature for energy demand. We first train a univariate model with the group attention layers switched off. Then we freeze the model and only train the group attention layers and a few small covariate layers on series with covariates. This makes training cheaper, lets us iterate on covariates without retraining everything, and keeps univariate forecasts the same.

Training Efficiency

Increased Graded Signal

The first design decision around training efficiency was to maximize the graded signal of each batch: the number of values that carry a loss. This reduces the compute needed for training and speeds up experiments. To get there, we adopted a modified version of the Contiguous Patch Masking strategy from the TiRex paper. We differ from TiRex in two ways: we use more diverse masking strategies inside the series, and we always hide patches at the end of every sequence. The hidden patches at the end of the sequence keep the overall task aligned with forecasting, and the diverse masking strategies make the model more robust to realistic data, such as outages, sensor glitches, and maintenance windows (Fig. 3).

Fig. 3: Example training masks. Gray gaps are hidden.

Full size

Sequence Packing

The second strategy is sequence packing. It concatenates multiple series into a single row to reduce padding waste. By packing shorter series into full training rows, we increase GPU efficiency and throughput during training.

Fig. 4: Padded rows (left) vs. packed rows (right).

Full size

Next-Patch Head

The third strategy is a next-patch head: a single linear layer that predicts the next patch, hidden or not, from every visible patch. We chose this lightweight design to force the backbone to learn richer representations inside its embedding space, rather than relying on extra parameters in the head.

Together, these changes multiply the number of graded values in each batch, as the figure below shows.

Fig. 5: Graded values per batch, by change.

Full size

Data and Training

Training Data

We train on a mixture of real and synthetic time series. To prevent leakage, the periods that fev-bench, GIFT-Eval, and M4 score are cut from our corpus. Our synthetic data comes from TCM, our temporal causal model prior: small networks that turn simple ingredients like trends, seasonal cycles, and sudden jumps into complex, related series. Because the series come from causal networks, we can control how covariates shape a target. Finally, we add real-world flaws such as missing values and irregular sampling to make the model more generalizable.

Loss Function

We train with a quantile (pinball) loss over nine quantile levels (0.1, 0.2, …, 0.9). On top of this, we apply spread-based weighting: each predicted value is weighted by the width of its predicted quantile interval, so a wide interval contributes less to the loss. This shifts the training signal away from noisy, unpredictable values and toward ones the model can actually learn from (Fig 6).

Fig 6: Narrower quantile bands mean more weight in the loss.

Full size

Get Started

TimeH2O-v1 and TimeH2O-v1-tiny are on Hugging Face, with weights for non-commercial use. The inference code is open source under Apache-2.0 as the h2ofm package: pip install h2ofm.

References

• Toto 2.0: Time Series Forecasting Enters the Scaling Era
• Chronos-2: From Univariate to Universal Forecasting
• TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning
• GIFT-Eval: A Benchmark for General Time Series Forecasting Model Evaluation
• fev-bench: A Realistic Benchmark for Time Series Forecasting
• It's TIME: Towards the Next Generation of Time Series Forecasting Benchmarks