Should We Build a Foundation Model in a Forest of Trees? A Pragmatic Guide to Time Series Foundation Models

Author’s Preface: This post was written in November 2026. I followed the seminal Transformer paper (Attention Is All You Need) back when it first appeared in 2017, though like most people at the time, my focus was largely on machine translation and NLP Seq2Seq tasks. It was not until late 2022, when ChatGPT burst onto the scene and ignited the generative AI revolution, that a simple intuition truly struck me:

“If a large language model (LLM) can read all the text in the world and generate answers to arbitrary questions, why couldn’t there be a time series foundation model that reads all the temporal data in the world and forecasts future trajectories directly?”

Yet in day-to-day industrial practice—across my earlier posts on Prophet, tree models (From a Single Tree to a Forest), and graph neural networks (GNNs)—the workhorse carrying billions of dollars in commercial transactions has remained a dense, resilient, and deeply pragmatic “forest” of Gradient Boosted Decision Trees (GBDTs). How far has that dream of a universal time series foundation model actually come? And should we really clear-cut this thriving forest to erect an optical “Tower of Babel” in its place?

Far from an exhaustive academic survey, this post is a set of reading notes born from grappling with that long-standing question across recent papers and technical reports, paired with practical reflections from hands-on demand forecasting. During drafting and editing, I also used AI tools including Gemini 4 Argon, Claude, and ChatGPT/Codex as editorial assistants; key metrics have been cross-checked against primary literature, and vendor-reported benchmarks are explicitly noted. Because this field moves rapidly and individual perspectives are inherently bounded, any corrections or peer feedback are warmly welcomed.

TL;DR in Three Points:

1. Repurposing language models offers questionable ROI: Rigorous ablation studies show that feeding numerical series into GPT-2 or LLaMA is rarely cost-effective—stripping away the multi-billion-parameter LLM backbone leaves predictive accuracy essentially unchanged while speeding up training from several-fold to over a thousand-fold, proving that gains stem primarily from patch tokenization and instance normalization;
2. Native time series foundation models show genuine promise: Compact models pretrained from scratch on physical temporal data (such as Chronos-2 and TimesFM 3.0, sized at tens to hundreds of millions of parameters) represent a vastly more practical path, dramatically lowering the burden of manual heuristic rules in cold-start launches, promotional extrapolation, and dynamic soft proxies;
3. Tree models remain the core baseline; foundation models serve as precision complements: In production, GBDT decision trees remain the unassailable workhorse for the overwhelming majority of mature business volume; using native foundation models as zero-shot shadow challengers or extracting their embeddings as prior features for tabular models is the most grounded engineering strategy today.

Over the past two years, machine learning conferences (such as NeurIPS, ICLR, and ICML) have showcased an intriguing trend: a steady stream of research grafting pretrained text models such as GPT-2 or LLaMA onto continuous time series forecasting (prominent examples include OneFitsAll/GPT4TS [1], Time-LLM [2], and CALF [3]), heralded under banners like Cross-modal Transfer and Prompt Reprogramming.

The theoretical narrative sounds seductive: if a language model has absorbed sequential reasoning and universal priors from trillions of text tokens, adapting it to predict electricity demand, weather patterns, or retail inventory should deliver an effortless leap forward.

Yet when these designs exit the academic ivory tower and confront real-world unit economics (ROI) on production lines, engineering teams are met with hard tradeoffs. Does chasing fractional leaderboard gains justify keeping tens of thousands of dollars in high-end GPUs on permanent standby, while enduring hundreds to thousands of times greater inference latency and memory bloat? More critically: how much of that reported performance genuinely originates from the expensive “language model” itself? At NeurIPS 2024, a rigorous ablation study by Tan et al. [4] delivered an eye-opening reality check: removing the language backbone entirely and replacing it with a minimalist linear projection or shallow network produced virtually identical accuracy. The impressive gains trumpeted in these papers were largely an illusion created by upfront patch encoding and normalization.

Simultaneously, the time series field has undergone its own quiet revolution. Pretrained native time series foundation models (TSFMs)—spearheaded by Chronos (Amazon) [5], TimesFM (Google) [6], Moirai (Salesforce) [7], and second-generation releases such as Chronos-2 [8] and TimesFM 3.0 [9]—are rapidly defining the mainstream. Built from scratch on vast corpora of heterogeneous temporal data without NLP vocabulary baggage, their parameter counts sit cleanly between tens of millions and a few hundred million parameters—fundamentally different from massive general-purpose text LLMs.

Any rigorous engineering discussion of “foundation models for time series” must immediately distinguish between these two paradigms:

• Repurposed LLM adaptation: The primary subject of engineering skepticism in this guide. The language backbone offers unproven transfer gains on pure numerical series while imposing exorbitant computational and operational overhead;
• Native time series foundation models: Purpose-built temporal foundation models pretrained across diverse domains. They deliver verifiable cold-start generalization and exogenous covariate alignment (such as promotions and calendar holidays), warranting serious, objective engineering evaluation.

Across mission-critical production pipelines, Gradient Boosted Decision Trees (GBDTs) like XGBoost and LightGBM have long grown into a dense, battle-tested “forest”. As foundation models arrive at the enterprise gate, the ultimate question for architects and algorithms teams is the one posed in our title: should we build a time series foundation model deep inside this thriving forest of trees?

Unpacking that architectural dilemma comes down to three concrete operational questions:

1. Why is the accuracy ceiling in production forecasting dictated primarily by data observability (such as repairing stockout censoring) rather than model capacity?
2. Why do classical baselines—the GBDT “forest” and statistical tools like Prophet—remain the most cost-effective workhorses for high-throughput, seasonal, tabular workloads?
3. When is a native time series foundation model truly worth deploying, and where should production teams draw the boundary between the forest and the foundation?

The bottom line: for the vast majority of routine enterprise forecasting, there is no need to clear-cut an existing GBDT forest, nor should teams reflexively wrap LLMs in adaptation layers; rigorous data engineering paired with robust tree baselines remains the highest-ROI strategy. Yet in the historical blind spots where decision trees struggle most—cold-start launches, sparse-history items, and cross-domain zero-shot forecasting—compact, natively pretrained time series foundation models provide an extraordinarily valuable new weapon for the engineering arsenal.

1. Repurposing Language Models: An Engineering Reality Check #

To understand why academic benchmarks and production engineering often diverge, it helps to trace the four generations of time series forecasting architectures:

1.1 Four Eras of Forecasting Evolution #

1. Era 1: Classical Statistical & Econometric Models (still widely deployed in production):
   • Core representatives: ARIMA, exponential smoothing (ETS), Theta methods, and Prophet [10].
   • Core mechanism: Operating on individual scalar series, these methods apply differencing for stationarity alongside state-space equations or explicit additive decomposition: \(y(t) = g(t) + s(t) + h(t) + \beta X(t)\). They rely on autocorrelation and stable periodicity assumptions. Prophet decomposes a series into trend \(g(t)\), Fourier seasonal terms \(s(t)\), and holiday priors \(h(t)\), offering transparent component-level attribution for business stakeholders. Because each series is fitted independently, however, CPU wall-clock time accumulates rapidly across hundreds of thousands of SKUs, and models cannot share cross-series patterns.
2. Era 2: Feature Engineering & Gradient Boosted Trees (2015–2020):
   • Core representatives: Gradient Boosting Decision Trees (GBDT), led by LightGBM [11] and XGBoost [12].
   • Core mechanism: Time series forecasting is cast as a supervised tabular regression problem over flattened panel data. Engineers construct lag features, rolling window statistics (such as 7-day and 30-day moving averages or exponential moving averages), and price-calendar interactions, training a shared model across thousands of series on standard CPU clusters.
3. Era 3: Task-Specific Deep Learning & Temporal Transformers (2018–2023):
   • Recurrent and hybrid architectures: Early deep models used RNN / LSTM cells, leading to autoregressive probabilistic frameworks like DeepAR and multi-horizon architectures such as TFT (Temporal Fusion Transformer) [13], which combines gating networks, variable selection, and attention for interpretable forecasting.
   • The Linear vs. Transformer debate: In 2023, Zeng et al. (AAAI 2023) [14] introduced DLinear, showing that a single linear layer with trend-seasonal decomposition could outperform complex Transformers on standard benchmarks. All-MLP architectures like TSMixer [15] and dense encoders like TiDE [16] reinforced the strength of lightweight networks. In parallel, PatchTST [17] popularized subseries patching and Channel Independence (CI), while iTransformer [18] inverted the attention mechanism to operate across variates rather than time steps. Notably, Autoformer, TimesNet, iTransformer, and the widely used Time-Series-Library (TSLib) benchmark suite all originated from the same research group at Tsinghua University. Spanning years of deep technical accumulation, the team completed the full journey from specialized deep models to native foundation models, later introducing Timer [19] and Sundial (日晷) [20]—whose name, meaning a classical timekeeping sundial, naturally evokes the iconic sundial engraved with the motto “Deeds, Not Words” (Xing Sheng Yu Yan) on the lawn before Tsinghua’s grand auditorium, symbolizing both the measurement of time and the inheritance of temporal scholarship.
4. Era 4: The Foundation Model Divergence (2023 to Present):
   • Path A (Repurposed LLM adaptation): Represented by Time-LLM [2], OneFitsAll (GPT4TS) [1], and CALF [3]. These methods patch or quantize numerical series, align them with word token embeddings via linear projections or cross-attention, and pass them through frozen or LoRA-tuned text backbones (GPT-2 or LLaMA) to predict future steps.
   • Path B (Native time series foundation models, TSFMs): Represented by Amazon’s Chronos-2 [8], Google’s TimesFM 3.0 [9], and Salesforce’s Moirai [7]. Dispensing with text vocabularies and discrete binning, recent native models project continuous patches directly into purpose-built Transformer backbones pretrained on massive real and synthetic temporal corpora, supporting multivariate interactions, exogenous covariates, and probabilistic quantile outputs out of the box.

Intuitive Parameter Breakdown: How Big Are 128M and 330M?

Engineers new to foundation models are sometimes put off by parameter counts in the “hundreds of millions.” In practice, a model parameter is simply a floating-point weight stored in memory:

• Native time series foundation models are remarkably compact: Tsinghua’s Sundial Base (128M parameters), stored in standard 32-bit floating point (FP32, 4 bytes per parameter), occupies roughly 513 MB on disk—the exact size of its model.safetensors checkpoint on Hugging Face. That is roughly equivalent to 100 MP3 tracks or less than a single 700 MB CD-ROM, fitting comfortably into standard laptop memory. Google’s TimesFM 3.0 has ~330M parameters (~1.32 GB on disk, smaller than a standard-definition movie; note that its PyTorch weights are released under the non-commercial timesfm-non-commercial-license-v1.0 license). This compact footprint explains why Sundial’s authors could run zero-shot inference in a few hundred milliseconds on an Apple M1 MacBook CPU with 16 GB unified memory.
• Repurposed language models are orders of magnitude heavier: By contrast, an adaptation pipeline built on LLaMA-7B (such as Time-LLM) carries 6.6 billion parameters—nearly 27 GB in FP32, or more than 52 times the size of Sundial—while 100B+ text LLMs require hundreds of gigabytes of distributed VRAM.

Static disk size vs. runtime memory: During live inference, memory consumption exceeds static weight size because the runtime must allocate KV caches, intermediate activation tensors, and batch buffers. Even with runtime overhead, however, serving a 100M–300M native time series model on a single mid-range GPU or modern CPU node is vastly more economical than hosting a multi-billion-parameter language model.

1.2 Data Observability Sets the Accuracy Ceiling #

Why do intricate deep architectures often beat simple baselines by wide margins on academic datasets, yet stall when deployed on enterprise production data? The root cause rarely lies in network capacity; it lies in data observability and aleatoric uncertainty:

1. Irreducible noise and unrecorded covariates: In retail demand, equipment telemetry, or financial markets, the dominant drivers of variance—competitor flash sales, sudden supply chain bottlenecks, or unlogged operator interventions—are frequently absent from the historical database. When key causal drivers are unobserved, scaling a backbone from 10 million to 7 billion parameters only encourages the network to overfit historical noise.
2. Stockout censoring (Censored Demand): In retail and supply chains, transaction logs record observed sales rather than true unconstrained demand. If a popular item sells out at 2:00 PM, point-of-sale systems log zero sales for the rest of the day. Feeding raw, unadjusted sales directly into a foundation model trains it to mistake supply stockouts for demand collapses, systematically under-forecasting peak demand.
3. Flaws in legacy academic benchmarks: In the ICML 2026 benchmark study TIME (Qiao et al.) [21], a comprehensive audit of 30 widely used public datasets (including ETT [22], Traffic, and Weather) exposed structural flaws in standard academic evaluations:
   • Data corruption and leakage: Several staple datasets contain unflagged missing values, erratic timestamp drifts, and overlapping train-test windows that inadvertently reward memorization;
   • Artificial forecast horizons: Standard benchmarks routinely require predicting 720 future steps from 96 historical steps on ETT (Electricity Transformer Temperature). In real substation operations, oil temperature is driven by ambient weather forecasts and immediate load schedules—no utility operator forecasts hourly transformer temperatures 30 days out from temperature curves alone.

In real-world production systems, investing engineering budget in data observability governance—such as imputing stockout-censored sales, aligning asynchronous timestamps, and building reliable promotional calendars—consistently delivers far higher business ROI than blindly swapping in heavier deep models over noisy tables.

1.3 Backbone Ablations and the Compute Ledger #

Does a language model’s text pretraining actually help numerical forecasting?

At NeurIPS 2024, Tan et al. [4] answered this question with a controlled ablation study across leading LLM-adaptation methods (OneFitsAll [1], Time-LLM [2], and CALF [3]). Keeping the input patching, normalization, and output heads intact, they compared the original pretrained LLM backbones against three stripped-down variants: replacing the LLM with a single attention layer (LLM2Attn), replacing it with a single Transformer block (LLM2Trsf), or deleting the backbone entirely (w/o LLM) so patches pass straight to a linear layer across 8 benchmark datasets (encompassing long-sequence forecasting tasks) over 40 distinct evaluation scenarios:

• Accuracy without the LLM: Across 40 evaluation settings (8 datasets × 5 forecast horizons), the lightweight LLM2Attn, LLM2Trsf, and w/o LLM variants matched or beat the full pretrained language models in 35/40, 31/40, and 29/40 settings, respectively.
• The compute ledger: On the Weather benchmark, the original Time-LLM (built on LLaMA-7B, 6,642M parameters) required 3,003 minutes (~50 hours) for a single training run. Removing the LLaMA backbone (w/o LLM) shrank the model to 0.198M parameters (<200K) and finished training in 1.91 minutes—a >1,500x training speedup without any degradation in test accuracy. Even for the lighter GPT-2-based OneFitsAll (86M parameters, 152 minutes of training, MAE 0.272), stripping the backbone reduced the model to 4M parameters, cut training time to 16 minutes, and lowered MAE to 0.254. Across all tested architectures, pretrained LLMs added 1.2x to 2.3x inference latency without delivering commensurate accuracy gains.
• Input shuffling and random weights: When the authors randomly initialized the LLM weights from scratch instead of loading pretrained text checkpoints, performance barely changed—in fact, on CALF, randomly initialized weights beat pretrained GPT-2 weights in 8 datasets versus 3. Furthermore, shuffling the temporal order of input tokens degraded LLM-adapted models no more than simple baselines, confirming that frozen text backbones do not contribute unique sequential reasoning to numerical time series.

Where, then, did the reported gains in those papers come from? The ablations pinpoint the real contributors: subseries patching, Reversible Instance Normalization (RevIN [23], which strips per-window mean and variance before inference and restores them at output to counter distribution shift), channel independence, and the trainable input/output linear projections. Providing solid ablation proof that cross-modal LLM wrappers provide no unique edge on numerical benchmarks, this work applied a well-timed brake to an overheating trend.

1.4 Why Production Teams Still Default to GBDT #

If grafting text LLMs onto numerical series fails the cost-performance test, what actually powers large-scale enterprise forecasting? In most production environments, the workhorse remains Gradient Boosted Decision Trees (GBDT):

1. Proven mainstream leadership in hierarchical retail (M5): In the landmark M5 Forecasting Competition [24] covering 42,840 hierarchical Walmart retail series, the vast majority of top-finishing solutions relied on LightGBM [11] or XGBoost [12]. Paired with rolling window aggregations, lag features, and price-calendar interactions, GBDT models train rapidly on commodity CPU clusters, outperforming both traditional econometric scripts and early deep learning architectures.
2. The resilience of simple linear baselines (TFB): In the VLDB 2024 TFB benchmark (Qiu et al.) [25], which evaluated 35 methods across 8,068 univariate and 25 multivariate datasets under unified protocols, simple linear regression still outperformed many complex deep networks on noisy physical series such as wind power.
3. Understanding GBDT’s real production moat: On newer multivariate benchmarks such as fev-bench [26], an untuned default LightGBM baseline will trail state-of-the-art native foundation models in zero-shot accuracy. However, GBDT’s industrial moat was never built on untuned zero-shot leaderboards; it rests on microsecond single-row inference latency, low-cost CPU scheduling, and strict operational guardrails via monotonic constraints (monotone_constraints) and SHAP feature attribution.

At the same time, classical tools have well-known structural boundaries:

• Prophet’s scaling bottleneck: Prophet works well for low-frequency, high-priority aggregate series with strong holiday and seasonal priors (such as company-wide daily GMV). Across millions of granular Store-SKU series, however, fitting each curve sequentially on CPU creates a heavy scheduling bottleneck compared with batched GBDT pipelines.
• GBDT’s feature and extrapolation limits: Tree models depend heavily on manual feature engineering. Because decision trees partition feature space via orthogonal axis-aligned splits, they cannot naturally extrapolate beyond the historical training range, and they struggle when a newly launched SKU or sensor has zero historical lags.

Key Concept Clarifier:

• Subseries Patching: In NLP, each token represents a word or subword. If a time series model treats every single timestamp as an isolated token, sequence lengths explode and individual scalars carry no local waveform shape. Patching groups adjacent timestamps (e.g., 16 or 24 consecutive points) into a single vector, compressing sequence length while preserving local trend and volatility patterns.
• Ablation Study: Borrowed from experimental medicine, an ablation study removes or replaces one component of a system at a time to measure its true contribution. When replacing a multi-billion-parameter language backbone with a single linear layer leaves forecast error unchanged, the ablation proves the heavy backbone was not driving the performance.
• Aleatoric Uncertainty & Data Observability: Real-world operational systems contain irreducible randomness and unlogged events—such as an unannounced competitor discount or a local transit disruption. When those causal signals are absent from the input table (unobservable), increasing model parameter count tenfold cannot predict an event that left no prior trace in the data.
• Monotonic Constraints: A critical production guardrail in tree frameworks like LightGBM and XGBoost. Basic economics dictates that deeper price discounts should not reduce expected sales volume. Unconstrained nonlinear models can overfit local noise and predict lower sales at 40% off than at 30% off. Enabling monotone_constraints forces every tree split to respect directional domain rules.
• Choosing Between LightGBM and XGBoost: LightGBM and XGBoost usually deliver very similar predictive accuracy; LightGBM typically trains faster with a smaller memory footprint on CPUs, whereas XGBoost has a more mature GPU training ecosystem. In practice, teams should benchmark both on their own business data using the exact same feature set before standardizing.

2. The Rise of Native Time Series Foundation Models #

Once we set aside repurposed text LLMs, the spotlight shifts to the genuine breakthrough of the past two years: native Time Series Foundation Models (TSFMs).

Before evaluating specific architectures, it is worth clarifying a foundational question: what actually qualifies as a true “time series foundation model”?

2.1 Defining True Time Series Foundation Models #

While no single standards body polices the term, researchers and practitioners generally evaluate models against five core criteria:

1. Decoupled pretraining and deployment: Pretrained once at scale on heterogeneous temporal corpora, the model can be applied directly to unseen target datasets without retraining from scratch for each task. Looking at the historical progression:
   • Local models (e.g., ARIMA, Prophet): Fit a separate set of parameters to each individual time series;
   • Task-specific global models (e.g., DeepAR, DLinear, PatchTST): Share parameters across all series within a single dataset, but must be retrained whenever applied to a new dataset or domain;
   • Time Series Foundation Models (TSFMs): Pretrained once across domains and deployed directly in zero-shot mode across diverse unseen tasks.
2. Cross-domain, multi-frequency pretraining corpora: Training data spans energy, traffic, meteorology, healthcare, and retail across minute, hourly, daily, weekly, and monthly sampling rates rather than a single vertical.
3. Zero-shot and few-shot generalization: The model delivers competitive point and probabilistic forecasts on unseen series immediately out of the box, without gradient updates.
4. Unified task interface: The architecture accommodates flexible context lengths, forecast horizons, and variate dimensions; newer releases also ingest exogenous covariates and support anomaly detection or missing-value imputation.
5. Foundational does not mean massive: Effective time series foundation models are remarkably compact: Moirai 2.0 has just 11M parameters, TiRex has ~35M, Chronos-2 has 120M, and Google’s multivariate TimesFM 3.0 has 330M. They do not need the multi-billion-parameter scale of conversational LLMs.

Under these criteria, the current landscape falls cleanly into three categories:

• Native time series foundation models (TimesFM, Chronos family, Moirai, TiRex, Sundial): Inspired by Transformer architectures but pretrained from scratch on large-scale temporal data; these are the recognized mainstream TSFMs.
• Task-specific deep forecasting models (DLinear, PatchTST, iTransformer): Trained supervised on a target dataset and retrained for each new domain; these are specialized deep networks rather than foundation models.
• LLM-adaptation wrappers (GPT4TS, Time-LLM): Require dataset-specific training for their adaptation heads and prompts, while ablations show their frozen language backbones add no irreplaceable temporal signal; calling them “time series foundation models” remains contentious.

2.2 Mainstream Architectural Lineages #

Unlike LLM-adaptation wrappers, native TSFMs are designed and pretrained from scratch on large-scale real and synthetic time series with temporal loss functions. Across the industry, three primary paradigms have emerged for continuous representation and future trajectory generation:

1. Discrete binning with cross-entropy token modeling: Quantizing scaled floating-point values into discrete vocabulary bins and training with standard autoregressive cross-entropy loss (e.g., first-generation Chronos);
2. Continuous patch embeddings with parametric distribution heads: Projecting continuous patches into latent vectors and predicting parameters of fixed distributions such as Gaussian, Student-t, or mixture models, or fixed quantile sets (e.g., TimesFM, Moirai);
3. Continuous patch embeddings with generative flow matching: Pairing continuous patch representations with Flow Matching (an ordinary-differential-equation generative framework that learns continuous transport paths from noise to data trajectories without assuming a fixed parametric distribution family), enabling direct multi-step trajectory sampling (e.g., Sundial [20]).

The major native foundation model families include:

• Chronos Family (Amazon):
  • Chronos-1 [5]: Applied numerical scaling and discrete binning (Scaling & Binning) to quantize continuous values into 4,096 tokens, pretraining a T5 architecture on ~84 billion observations;
  • Chronos-2 [8]: Replaced discrete binning entirely with continuous patch projections and an encoder-only architecture, adding native support for multivariate cross-series interactions and exogenous covariates (such as promotions and holiday calendars);
• TimesFM Family (Google) [6][9]:
  • TimesFM 1.0/2.0: Used a decoder-only continuous patch architecture focused on high-throughput univariate zero-shot forecasting;
  • TimesFM 3.0 (released by Google in August 2026) [9]: Marked the jump from univariate to native multivariate forecasting. With 330M parameters pretrained on over 1 trillion real and synthetic time points, it alternates causal temporal attention (Causal Temporal Attention) with full variate attention (Full Variate Attention), natively supporting both past-observed covariates (Past-observed Covariates) and future-known covariates (Known Future Covariates); in Google’s published benchmarks, it ranks near the top across GIFT-Eval [27], fev-bench [26], and TIME [21];
• Moirai Family (Salesforce) [7]: First-generation Moirai used a masked encoder architecture (14M–311M parameters) with any-variate attention; Moirai 2.0 streamlined the design into an 11M-parameter decoder-only model built for fast universal inference;
• Sundial Family (Tsinghua University, ICML 2025 Oral) [20]:
  • Uses a decoder-only continuous patch architecture (patch length 16) trained with TimeFlow Loss (a flow-matching generative objective) to sample continuous future trajectories directly from standard Gaussian noise, eliminating the constraints of rigid parametric distribution families;
  • The base model has 128M parameters (alongside 32M and 444M variants), supports a 2,880 context length, and was pretrained on roughly 1 trillion time points;
  • Supports univariate forecasting only (no exogenous covariates); ranked near the top on GIFT-Eval [27] MASE at release (based on author evaluations, with the public leaderboard subsequently updated by newer models);
  • Critical caveat: Sundial’s zero-shot evaluations on TSLib still rely heavily on legacy benchmarks such as ETT [22], ECL, and Weather; those scores should be weighed alongside the structural benchmark critiques raised by TIME [21].

To highlight the structural differences between repurposed language models and native time series foundation models, the diagram below compares both paths across five core dimensions: input representation, token/patch mapping, pretrained backbone, output distribution, and production trade-offs.

The following chart illustrates this architectural split from top to bottom:

2.3 Strengths: Cold Starts and Exogenous Covariates #

In production engineering, native TSFMs are not meant to rip and replace mature GBDT pipelines; instead, they solve three problems where classical tools struggle most:

1. Cold-start forecasting for new SKUs and sensors (Cold-start Zero-shot):
   • Retail chains launch new items daily, and industrial plants regularly add new sensors with little or no history. When a series has only a dozen historical observations, GBDT cannot populate 7-day or 30-day lag features, and statistical models cannot fit stable seasonal curves. Native models like Chronos-2 and TimesFM 3.0 infer reasonable probabilistic trajectories immediately from short context windows without fine-tuning.
2. Native alignment of heterogeneous covariates:
   • Classical univariate models struggle to ingest rainfall, holiday calendars, and promotional depths simultaneously. Chronos-2 and TimesFM 3.0 integrate dynamic covariates directly into their attention layers. In the Chronos-2 authors’ retail and energy evaluations, covariate-aware inference significantly outperformed both univariate Chronos-2 and covariate-capable baselines such as TabPFN-TS.
3. Simplified MLOps and unified inference APIs:
   • Maintaining tens of thousands of individual Prophet or ARIMA states across categories imposes heavy scheduling overhead. A single 330M-parameter model like TimesFM 3.0 (for example, exposed via BigQuery AI.FORECAST) can serve heterogeneous series across an organization through one standardized microservice API.

2.4 Operational Limits and Engineering Trade-offs #

Even native time series foundation models carry clear operational trade-offs in production:

• The conservative mean trap (Conservative Mean Prediction): Qualitative analyses in the TIME benchmark [21] show that when confronted with noisy or volatile industrial series, many foundation models hedge their statistical error metrics (such as MAE and MSE) by outputting an overly smooth horizontal line near the mean. While a flat line avoids catastrophic mathematical penalties, under-forecasting peak demand triggers massive stockouts in retail replenishment and blinds operators to critical equipment anomalies, offering little actionable value for real-world operations.
• Hardware footprint and throughput ledger: According to the Chronos-2 paper, a single NVIDIA A10G GPU processes roughly 300 series per second (with its smaller 28M variant running faster and supporting CPU inference). While vastly lighter than a 7B language model, 300 series/sec on a GPU still requires careful cost accounting when compared against LightGBM scoring hundreds of thousands of rows in microseconds on existing CPU nodes.

Key Concept Clarifier:

• Zero-Shot Generalization: Deploying a pretrained model directly onto series it never saw during training (such as a newly installed factory pump or a newly launched yogurt SKU) and obtaining competitive forecasts without local training or gradient updates.
• Exogenous Covariates: External variables outside the target series that help explain its movements. Covariates split into future-known covariates (scheduled promotional discounts, upcoming public holidays) and past-observed covariates (yesterday’s actual rainfall or historical foot traffic).
• Continuous Patching vs. Discrete Binning: First-generation Chronos imitated NLP models by quantizing continuous floating-point values into 4,096 discrete vocabulary buckets (Bins). Chronos-2 and TimesFM dropped discrete binning in favor of linear patch projections that map contiguous slices of floats directly into embedding vectors, preserving numerical geometry while cutting decoding latency.
• Causal Temporal vs. Full Variate Attention: The core building block of native multivariate models like TimesFM 3.0. Because time flows forward, attention along the time axis uses a causal mask (Causal Temporal Attention) so future points cannot leak into the past. Within the same timestamp, however, different variables (temperature, wind speed, power load) interact mutually with no fixed ordering, so the model applies unmasked bidirectional attention across variates (Full Variate Attention). Alternating both layers captures temporal causality and cross-variable coupling together.

3. Micro Showdown: Single-Store Cold-Start Forecasting #

Thought Experiment Context & Disclaimer: The retail scenario outlined in this and subsequent chapters—a stylized nationwide Australian omnichannel chain with 200 stores, 2.3 million Store-SKU combinations, 4.19 billion transaction records, fresh produce/bakery departments, and Sydney/Melbourne transshipments—is strictly a stylized thought experiment constructed from public retail domain knowledge and academic benchmark practices, with its merchandise hierarchy (State -> Category -> Sub-category -> Store -> SKU) and daily panel structure modeled after the public Walmart M5 Forecasting Competition [24] paradigm. All figures, SKU counts, table sizes, promotional mechanisms, and replenishment assumptions are intended solely as realistic stress-test baselines for technical comparison. They do not represent the actual metrics, confidential data, production architecture, or commercial decisions of any real-world enterprise, nor do they constitute an operational evaluation of any specific retailer.

To see how these competing paradigms behave in day-to-day production, this section walks through a concrete thought experiment set in the Australian Omnichannel Retail Market:

Unified Commercial Scenario: Suppose a national supermarket chain operates 200 physical stores across five Australian states (New South Wales NSW, Victoria VIC, Queensland QLD, Western Australia WA, and South Australia SA). Products are organized under a strict merchandise hierarchy:

• Network-wide category tree: 10 primary Departments / Categories (Fresh Produce, Food & Beverage, Household, etc.) and 50 secondary Sub-categories (Ready-to-Drink Tea, Carbonated Soft Drinks, Premium Chilled Dairy, etc.), drawing from an active SKU Master Pool of roughly 25,000 items.
• Assortment depth by store format: Because trade-area demographics and floor space vary across locations, active SKU counts differ sharply by store format:
  • 50 community convenience stores (~4,000 active SKUs each);
  • 100 standard suburban supermarkets (~10,000 active SKUs each);
  • 50 large hypermarket flagships (~22,000 active SKUs each).

Summing across all 200 stores (\(50 \times 4000 + 100 \times 10000 + 50 \times 22000\)) yields roughly 2.3 million Store-SKU series that require independent forecasts. Over five years (1,825 days) of historical transactions stored in the enterprise data lake, the daily panel table reaches 2.3 million × 1,825 ≈ 4.19 billion rows.

With the Chinese Mid-Autumn Festival approaching, headquarters launches a network-wide demand forecast and stock allocation plan for newly opened stores in high-density Asian communities (such as Sydney and Melbourne) alongside festive specialty aisles across mainstream stores (featuring limited-edition mooncakes, high-margin ready-to-drink teas, and festive fresh produce).

This campaign confronts the forecasting and supply chain teams with two distinct challenges:

• Micro execution: How to produce an accurate cold-start forecast for a newly launched specialty SKU at a single store with minimal history and stepped discounts?
• Macro architecture: How to architect an enterprise forecasting system across a multi-state hierarchy spanning 2.3 million Store-SKUs and 4.19 billion rows?

The analysis begins at the micro level, examining how classical models and native foundation models tackle single-item cold starts.

3.1 Business Context: Cold-Start Specialty Assortment #

Suppose a newly opened store in a high-density Asian precinct in Sydney launches an exclusive, high-margin festive beverage (a brand-new SKU) that has accumulated only 14 days of sales history. Over the upcoming 7-day forecast horizon, the store plans a stepped discount campaign leading up to Mid-Autumn Day. The target variable is daily unit sales, paired with two known future covariates: planned discount depth and a public holiday indicator.

Confronted with this setup, each modeling paradigm exhibits distinct operational behavior.

3.2 Classical Methods: Prior Constraints and Proxy FSMs #

1. Prophet’s Prior Shrinkage Under Cold-Start Covariates #

Prophet models a series via explicit additive decomposition: \(y(t) = g(t) + s(t) + h(t) + \beta X(t)\).

• The bottleneck: Across the first 14 days, the new SKU sold mostly at full price, leaving the discount covariate \(X(t)\) with almost no historical variance. With so few observations and minimal variance, the posterior estimate of the regression coefficient \(\beta\) is pulled back toward zero by prior shrinkage.
• The consequence: When a 50% discount arrives two days out, Prophet cannot amplify the promotional lift because \(\beta\) is under-estimated, outputting a flat curve that merely extrapolates the mild 14-day baseline trend.

2. GBDT’s Missing Lag Features and Extrapolation Limits #

Experienced engineers typically train a global tree model (such as LightGBM) across thousands of mature SKUs so new items can inherit store-wide promotional elasticity. In production, however, this runs into three hurdles:

• Missing lag features at inference time: With only 14 days of history, the new SKU lacks the lookback window needed to populate 7-day rolling means, 14-day exponential moving averages (EMAs), or 30-day lags, leaving key tree features as NaN or noisy short-window estimates.
• The tree extrapolation barrier: Standard decision trees partition feature space via orthogonal axis-aligned splits and output piecewise constant leaf values. If historical discounts in the training set never exceeded 30% off, a 50% discount falls into the outermost leaf and gets capped at the 30% volume level rather than extrapolating linearly (Note: In practice, LightGBM’s linear_tree mode fits local linear models in the leaves and can be paired with monotone_constraints to partially mitigate this clipping; however, because cold-start records are severely sparse, fitting local linear regressions in terminal leaves still frequently suffers from numerical instability or regression collapse).
• Look-alike mapping and state-machine overhead: The classic industrial fallback is Look-alike / Proxy Mapping—assigning a mature proxy SKU via category rules or attribute clustering and borrowing its historical curve. Maintaining this in production requires a hand-tuned Finite State Machine (FSM) that defines exactly how proxy weights decay after Day 7, Day 14, and Day 30. Hard switches create step-function jumps in replenishment orders, while soft blends require ongoing manual calibration.

3.3 Foundation Models: Cross-Series Attention and Soft Proxies #

Native time series foundation models (such as Chronos-2 and TimesFM 3.0) offer a more direct end-to-end mechanism. Chronos-2 addresses cold starts through Group Attention: at inference time, the new SKU’s short sequence is grouped alongside several mature proxy SKUs from the same store into a single input batch. By computing attention along both the time axis and the cross-series group axis, the cold-start series directly absorbs promotional response patterns from mature items in latent space.

This group-attention architecture removes the single-series isolation bottleneck. During inference, the target cold-start series, mature look-alike trajectories, and known future covariates are aligned into a multi-series tensor group. Passing through alternating temporal attention and cross-series group attention, the model outputs multi-quantile probability distributions that capture upside promotional uncertainty.

The diagram below traces this end-to-end tensor alignment and dual-attention flow:

This design brings three practical engineering benefits:

1. No manual lag feature construction: Continuous patch inputs ingest the 14-day history directly without requiring precomputed 30-day rolling windows.
2. Automated “soft proxy” blending: Group attention acts as a differentiable generalization of industrial look-alike mapping—rather than hard-coding a static 1-to-1 proxy rule, attention weights dynamically borrow historical trajectory priors from multiple mature reference series according to temporal similarity.
3. Eliminating rigid hand-coded state machines: As the new item accumulates real daily sales, under well-conditioned attention dynamics, self-attention can progressively shift weight toward its own observed trajectory in a continuous, differentiable manner—in principle smoothing the transition from prior-driven to history-driven forecasts.

One caveat remains essential: if a cold-start series has never experienced a major discount and is fed into the model in isolation without group reference series, no architecture can guess its 50% discount elasticity out of thin air; the foundation model outputs a wider prior probability interval rather than a guaranteed point oracle.

3.4 Micro Comparison Takeaways #

In single-store cold-start launches, native foundation models offer clear algorithmic agility through group attention and in-context learning, sidestepping missing lag features and brittle proxy state machines.

Yet tactical agility in “single-item combat” does not automatically guarantee victory in “large-scale campaign warfare.” Single-store launches represent only a small fraction of total enterprise retail volume. When the operational scope expands to 200 stores, millions of active Store-SKUs, and billions of rows of transaction history, system throughput, VRAM bottlenecks, and hierarchical reconciliation become the decisive architectural battlegrounds.

Key Concept Clarifier:

• Group Attention: Standard self-attention only links timestamps within a single series. Group attention packages multiple related series (such as a 14-day cold-start item and several mature items with full seasonal histories) into a shared forward pass, allowing attention to flow across series so the newcomer can borrow promotional trajectories in latent space.
• In-Context Learning: The ability of a pretrained foundation model to infer a sequence’s baseline level, noise scale, and periodicity directly from the conditioning context passed at inference time, without running backpropagation or updating weights.
• Look-alike Proxy Mapping & FSM: The classic production fallback for cold starts, pairing a new SKU with a mature reference SKU (e.g., mapping a new jasmine green tea to an existing oolong tea) and scaling its historical curve. To transition smoothly as real sales arrive, engineers maintain a Finite State Machine (FSM) that linearly decays the proxy weight from Day 8 to Day 30 before switching fully to the item’s own history.
• Intermittent & Zero-Inflated Demand: Prevalent in industrial spare parts and slow-moving long-tail SKUs, where demand is zero on most days of the year punctuated by isolated purchases. Training a standard MSE model on zero-inflated series pulls predictions toward a flat near-zero line; production systems typically rely on Croston’s method [28], Syntetos-Boylan approximations, or GBDT models trained with Tweedie or Zero-Inflated losses.

4. Macro Architecture: Commercial Panels and Partitioned Modeling #

Moving beyond a single cold-start SKU, enterprise architects have to design for the entire national matrix.

It is worth reiterating that the national commercial panel and architectural simulations explored in this chapter are strictly an industrial stress-test conducted within a thought-experiment framework—by positing an extreme scenario of 200 stores, 2.3 million Store-SKU entities, and 4.19 billion daily transaction rows over five years, the goal is to probe the physical limits and engineering resilience of competing paradigms under realistic enterprise data throughput.

Across a multi-state retail network facing such vast operational complexity, the three modeling paradigms represent data and allocate compute in fundamentally different ways:

4.1 Fundamental Divergence in Data Structures and Input Shapes #

At an architectural level, industrial time-series forecasting is fundamentally about mapping multi-dimensional physical transaction streams into in-memory data structures that hardware can process with maximum throughput. When scaling from prototype scripts to enterprise production, teams often overlook the stark generational differences in how competing paradigms ingest data:

• Classical Econometrics (Prophet): Shatters enterprise history into millions of isolated “1D scalar streams”;
• Machine Learning (GBDT): “Flattens” the entire commercial spacetime into a massive 2D feature matrix;
• Native Foundation Models (TSFMs): Aligns temporal series and covariates into continuous 3D tensors ([Batch, Time, Channel]).

The table below contrasts these three structural paradigms across data shape, tensor dimensions, cross-series linkage, and compute overhead for 2.3 million Store-SKU series:

Examining this input-output matrix reveals the hard compute trade-offs behind each tensor shape:

1. The single-series CPU bottleneck (Prophet): Fitting 2.3 million models independently—even across a 100-worker cluster—creates an orchestration nightmare while preventing models from sharing store-wide foot traffic or promotional elasticity;
2. The 2D memory monster (GBDT): Flattening 5 years of daily records into a 60-column panel produces a 4.19-billion-row table. Holding this table in memory requires expensive terabyte-scale infrastructure, and distributed shuffles become the primary latency bottleneck;
3. The 3D tensor context and throughput trade-off (Foundation Models): Streaming 3D tensors avoids maintaining a 4.19-billion-row wide table. However, in large-scale batched inference, balancing \(O(L^2)\) self-attention throughput overhead, standard pretraining context configurations, and diminishing marginal signal from distant history typically leads teams to truncate the input window to the most recent 512 or 1,024 steps (while in minute-level or high-frequency sensor workloads, long histories hit a hard VRAM wall directly).

This fundamental divergence in memory structures directly triggers three operational clashes between business needs and system architecture.

4.2 Three Core Clashes Between Business and Architecture #

1. Handling Entity IDs: Memorizing Baselines by ID vs. Inferring Level from Context
   • Why GBDT relies on IDs: Decision trees have no built-in temporal topology; they treat every row in the panel table as an independent sample. To distinguish high-volume flagships from small convenience stores, trees rely on categorical splits over Store_ID and SKU_ID to anchor each entity’s baseline sales volume.
   • Why foundation models ignore IDs: In zero-shot inference, a pretrained foundation model has no embedding entry for proprietary enterprise identifiers like Store_042 or SKU_98127. Passing high-cardinality categorical IDs adds no signal; instead, the model relies on in-context learning, reading the recent floating-point window directly to infer local mean level, variance scale, and seasonal shape.
2. Merchandising Blind Spots: Absolute Price Points vs. Relative Price Elasticity
   • In retail merchandising, “50% off” means very different things across price tiers: cutting a $5 bottled drink to $2.50 triggers immediate multi-pack impulse buying, whereas cutting a $100 premium health-supplement gift box or bulk infant formula tin to $50 drives planned pantry-stocking behavior subject to purchase limits and post-promotion demand pull-forward.
   • Where tree models excel: Decision trees naturally capture multi-way conditional interactions. By splitting on Category = Beverage and Retail_Price < 10, a tree easily fits distinct nonlinear price-elasticity curves for impulse items versus high-ticket items.
   • Where foundation and statistical models struggle: Many everyday grocery items keep a fixed regular shelf price for months at a time (zero local variance). For single-series Prophet models, although the underlying Stan optimizer places Gaussian priors (L2 regularization) on exogenous regression weights to prevent singular matrix crashes, a constant price column is collinear with the baseline intercept—its regression weight is shrunk toward zero by the prior, learning nothing about cross-item price tiers. Meanwhile, per-window instance normalization (Standard Scaling / z-score) inside foundation models strips away absolute price levels (turning both $100 and $5 into dimensionless relative numbers, while epsilon-smoothed constant series collapse to a flat zero vector), leaving self-attention blind to absolute price-point psychology unless cross-series price-tier covariates are explicitly injected.
3. Long Historical Lookbacks vs. Context Window Truncation: The \(O(L^2)\) Throughput Trade-off
   • Five years of clean daily transaction history are invaluable for modeling multi-year cycles or recurring climate anomalies such as Australian La Niña summers.
   • A GBDT pipeline can easily compute lag_365 (last year’s sales) and lag_730 (two years ago) across a CPU Spark cluster, distilling multi-year anchors into compact scalar columns before running a fast tree trainer.
   • Feeding ultra-long histories directly into a Transformer foundation model at inference time presents a different compute ledger: self-attention compute and memory scale quadratically (\(O(L^2)\)) with effective sequence length. For daily retail series, while 1,825 days compressed via patching (e.g., \(P=16\)) yields only ~115 patch tokens—well within single-series VRAM capacity—scaling batched inference and cross-series group attention across 2.3 million series makes long contexts significantly slower while introducing distribution-drift noise from years-old regimes; in minute-level store footfall or high-frequency industrial telemetry, five-year histories hit the physical GPU memory wall outright. Consequently, engineering teams following standard pretraining context configurations typically truncate input windows to the most recent 512 (or 1,024) days—meaning multi-year YoY and climate-cycle signals outside that window are lost unless separately extracted via feature engineering.

Confronted with the physical collision of entity ID memorization, nonlinear price elasticity, and long-horizon memory bottlenecks, deploying a single monolithic architecture is an engineering dead end. Neither relying blindly on memory-hungry full Transformer self-attention nor forcing single-machine RAM to ingest a multi-billion-row flat matrix can work. How, then, does industry actually break this deadlock?

4.3 Architectural Pivot: From Monolithic Models to Partitioning #

Faced with a 4.19-billion-row panel table, teams new to industrial forecasting often try to train a single monolithic GBDT model across the entire company.

In production, a single 4.19-billion-row model is both a compute bottleneck and an architectural anti-pattern. The proven industrial solution is Partitioned / Stratified Modeling—a direct and elegant resurgence of the classic “Divide and Conquer” algorithmic paradigm familiar to every computer science student: breaking one unwieldy 4.19-billion-row table along natural business hierarchy boundaries into 50 cross-series partitioned models grouped by State × Category.

Shifting from 1 monolithic model to 50 partitioned models turns an intractable cluster job into a manageable production system. Divide and conquer not only shatters single-node physical RAM limitations, but also strikes an optimal engineering balance across statistical accuracy, cluster scheduling, and business agility.

1. Why Dimensional Partitioning Works So Well #

Fitting one monolithic XGBoost or LightGBM model across 4.19 billion rows requires terabyte-RAM distributed clusters, and early tree splits waste depth separating unrelated regimes—such as winter sunscreen in Melbourne (VIC) versus down jackets in tropical Cairns (QLD)—injecting cross-category noise into leaf estimates.

Splitting the pipeline into 50 independent State × Category models delivers three major engineering advantages:

• Dramatic compute and memory scale-down: Dividing 5 states × 10 categories into 50 partitions reduces each model’s dataset to roughly 84 million rows (\(1/50\) of the total). At 84 million rows, engineering teams can drop expensive terabyte PySpark clusters and train native Python LightGBM models in parallel on standard 64 GB RAM cloud VMs, cutting infrastructure costs and training wall-clock time simultaneously.
• Focused domain splits: A tree model trained exclusively on New South Wales Beverages never has its splits diluted by Baby & Health or Pet Care; every level of the tree focuses on beverage weather sensitivity, pack-size elasticity, and promotional calendars.
• The MLOps sweet spot: Maintaining 2.3 million individual Prophet models is a scheduling nightmare, while relying on 1 monolithic global model creates a single point of failure (SPOF). Managing 50 partitioned models sits right in the operational sweet spot for modern MLOps: orchestrators like Airflow, Argo, or MLflow can schedule, monitor, canary-test, and roll back individual state-category partitions independently.

2. Granularity Dilemma: 50 State-Category vs. 250 State-SubCategory Models #

When partitioning a retail hierarchy, architects have to choose the right partition depth:

Architectural takeaway: 50 State-Category models provide the strongest production baseline. Keeping Sub-category inside each model as a categorical feature column preserves intra-department substitution and cannibalization effects; splitting all the way down to 250 sub-category models severs those competitive cross-elasticities across separate tables.

3. Resolving the Hidden Cost: Loss of Store-Level Foot Traffic Resonance #

Partitioning by category does introduce one subtle blind spot: losing store-wide foot traffic context.

• The failure mode: Once data is split by department, the Snack & Beverage model has no visibility into promotions running in the Fresh Seafood & Meat aisle of the exact same store (for example, a Mid-Autumn half-price flash sale on Australian rock lobster and Wagyu gift boxes). In physical supermarkets, a flagship fresh-food promotion drives store-wide family shopping trips that lift impulse snack and beverage sales. Physical category partitioning hides this cross-department halo effect.
• The engineering fix: Pre-Split Feature Injection: In the data warehouse ETL layer, before splitting the 4.19-billion-row base table into 50 category partitions, compute two store-level macro features across the full table:
  • Store_Total_Sales_Lag_1 (yesterday’s total store-wide revenue across all departments, acting as a foot-traffic proxy);
  • Store_Cross_Category_Promo_Index (an aggregate index of active promotional intensity across the entire store). Inject both macro columns into every one of the 50 partitioned tables.

With those two injected columns, even an isolated Snack & Beverage LightGBM model can sense store-wide traffic surges, eliminating the partition silo at negligible compute cost.

Through “50 State-Category partitions plus pre-split global feature injection,” the classical GBDT stack delivers an almost textbook engineering solution for macro volume. Yet an inevitable question arises: if partitioned decision trees are already so mature and cost-effective, is there any room left for native time series foundation models in such a massive production matrix? Where, precisely, does the real division of labor lie?

4.4 Real-World Trade-Offs: Foundation Models vs. Tree Partitions #

Stepping back from both the single-store cold start and the 200-store national matrix, the production division of labor between partitioned GBDTs and native TSFMs becomes clear:

1. Partitioned GBDT: The Mainstream Production Workhorse (Carrying the Vast Majority of Mature Volume)
   • For the 200 mature stores and tens of thousands of active SKUs that generate the vast majority of enterprise volume, 50 State-Category LightGBM models enriched with pre-split store traffic features remain the most reliable, cost-effective backbone.
   • Microsecond row latency, enforceable monotonic constraints (monotone_constraints), sharp price-tier interaction splits, and pure CPU batch throughput form a durable engineering moat.
2. Native Foundation Models: High-Leverage Solutions for Cold-Start Blind Spots
   • Native TSFMs are not meant to replace partitioned GBDTs on mature SKUs; they target the scenarios where tabular feature engineering breaks down—cold starts and zero-shot regime shifts:
     • New stores and new SKU launches: When a retailer opens a new store in a fast-growing suburb or introduces dozens of seasonal Mid-Autumn items with zero history, GBDT lag features are empty. Native foundation models like Chronos-2 and TimesFM 3.0 use in-context learning and group attention to output calibrated probability intervals from a handful of observations.
     • Low-overhead baseline exploration: When engineering bandwidth is tight, zero-shot TSFMs allow a team to stand up a multi-series probabilistic baseline in a few lines of code without building custom feature pipelines first.
3. Closed-Loop Lifecycle Handoff & Dual-Track Coexistence
   • In a mature retail forecasting architecture, the two paradigms are far from a zero-sum contest; they operate as a collaborative dual-track production pipeline:
     • Cold-Start Track: During Days 0–30 of a new SKU or store launch, a native foundation model provides zero-shot probabilistic forecasts to guide initial stock allocation;
     • Mature Production Track: Once 30+ days of transaction history accumulate and rolling lag features stabilize, the series is handed off smoothly to the partitioned GBDT production pipeline, capturing microsecond inference latency and enforceable monotonic constraints;
     • Latent Embedding as Prior Feature: In advanced engineering setups, teams can extract latent embeddings from the foundation model’s forward pass and inject them directly as continuous prior features into the GBDT wide table, giving decision trees a global waveform geometry prior when choosing splits. The frontier light from the foundation model illuminates the historical blind spots deep within the forest.

Yet moving from architectural blueprints to live production sign-off requires engineers to confront cold operational realities. Before scaling up a foundation model footprint, teams must balance five hard engineering ledgers.

4.5 Five Operational Concerns for Production Deployment #

Even where native foundation models excel, engineering leaders have to weigh five practical risks before expanding their footprint in production:

1. Reduced causal interpretability for business stakeholders:
   • Merchandising and supply chain planners expect clear attribution. Prophet decomposes forecasts into explicit additive terms ("+30 units from discount, +50 from holiday"), and LightGBM offers exact per-feature SHAP attributions. A 100M–300M parameter attention network remains a high-dimensional black box, making it harder to answer a buyer’s question when a forecast suddenly spikes.
2. GPU provisioning and unit-cost accounting:
   • LightGBM and Prophet run on commodity CPU nodes. Running batched foundation model inference at scale typically requires provisioning GPU instances (such as NVIDIA A10G) or paying per-call cloud API fees, which must be justified against microsecond CPU tree scoring.
3. Quadratic complexity on ultra-long contexts:
   • GBDT pipelines can reference 1-year and 2-year historical lags at virtually zero marginal training cost. In high-frequency industrial telemetry (such as 100 Hz vibration sensors), self-attention memory scales quadratically with context length, often forcing truncation before multi-week cycles are captured.
4. Intermittent and zero-inflated long-tail demand:
   • In spare parts or slow-moving retail SKUs where most days record zero sales, production pipelines rely on Croston’s method or Tweedie-loss GBDTs. Whether continuous patch tokens can reliably model sparse discrete demand pulses without collapsing toward zero remains an active engineering concern.
5. The maintenance gap between zero-shot and fine-tuning:
   • When a LightGBM partition drifts, adding two domain features and retraining on CPU takes minutes. If a zero-shot foundation model underperforms on proprietary enterprise series, fine-tuning it requires GPU training pipelines, checkpoint versioning, and regression checks against catastrophic forgetting.

Chapter Summary: Native time series foundation models (Chronos-2, TimesFM 3.0) deliver the highest ROI in cold-start launches, covariate-heavy short histories, lean teams without heavy feature-engineering bandwidth, and workflows with dedicated GPU budgets. For mature core assortments that require transparent attribution, strict CPU cost control, or sparse long-tail modeling, partitioned GBDT and classical statistical models remain the production bedrock.

Key Concept Clarifier:

• Partitioned / Stratified Modeling: The standard enterprise pattern for massive multi-hierarchy panel data. Splitting an unwieldy multi-billion-row table along geographic and merchandise boundaries (such as State × Category) reduces per-model memory footprints, sharpens tree splits around domain-specific patterns, and keeps MLOps scheduling modular.
• Cross-Subcategory Substitution Effect: Demand cannibalization across sibling subcategories within the same department—for example, a sharp increase in beef prices pushing shoppers toward chicken, or a deep discount on one cola brand pulling volume away from competing colas.
• Store-Level Global Context & Feature Injection: Computing store-wide total sales and aggregate promotional intensity in the data warehouse before splitting the base table by department, then injecting those macro columns into every partition so category models remain aware of store-wide foot traffic surges.
• SHAP Attribution: A game-theoretic feature attribution method based on Shapley values that decomposes an individual prediction into exact per-feature contributions—allowing an engineer to tell a category manager, “Of the +50 incremental units forecasted for Friday, +30 come from the 50% discount, +15 from the Mid-Autumn holiday, and +5 from sunny weather.”

5. Academic Exploration and the Engineering Bottom Line #

By now, readers will appreciate why industry remains cautious about deploying foundation models directly into production time-series pipelines. Yet dismissing earlier academic attempts to adapt pretrained language models as pointless would be equally shortsighted.

Erecting a monumental “Tower of Babel” in a dense and pragmatic “forest of decision trees” is never achieved overnight. Understanding why the research community began with language model fine-tuning—and learning to look past bolded leaderboard tables—honors the legitimate course of scientific inquiry while equipping production engineers with the discernment needed to defend system integrity.

5.1 The Value of Hypotheses Under Academic Constraints #

1. Testing a legitimate scientific hypothesis: Exploring whether high-dimensional representations learned by large language models over trillions of text tokens could serve as a transferable universal prior for physical time series was an entirely reasonable and intellectually bold hypothesis. Scientific progress relies on proposing hypotheses and testing them rigorously; disproving an assumption is as valuable as confirming it. The NeurIPS 2024 ablation study by Tan et al. [4] represents a healthy, peer-reviewed self-correction within the academic community;
2. Resource realities and pragmatic academic exploration: Pretraining a native foundation model from scratch carries a high barrier to entry. Tsinghua University’s Sundial [20], for instance, was trained on 32 NVIDIA A100 GPUs, and curating its \(10^{12}\)-time-point TimeBench corpus (spanning ERA5 global weather reanalysis [29] and diverse physical observations) built upon years of cumulative research infrastructure (TSLib, Timer series [19]) supported by national research programs. Such sustained compute and engineering capacity is not readily available to every university lab. For groups with modest computational resources, exploring parameter-efficient fine-tuning (PEFT), architectural adaptation, or lightweight baselines represents a pragmatic attempt to engage with the problem. Recognizing these differing resource baselines helps us evaluate academic contributions with greater nuance and fairness;
3. Where language models genuinely excel: Multimodal reasoning: As Tan et al. noted in their conclusion, a language model’s distinct value lies not in crunching raw numerical vectors, but in multimodal reasoning that fuses unstructured text with numerical series—combining historical sales curves with social media sentiment, industry research reports, breaking news, and earnings call transcripts.

5.2 Looking Past “Illusory SOTA”: Three Questions for Empirical Papers #

Candidly, reading many recent LLM-time-series papers feels like watching someone build an elaborate banquet around a single condiment—burning massive GPU hours and stacking intricate adaptation modules just to claim a single bolded SOTA row in a benchmark table.

This does not always reflect deliberate distortion: under-tuned classical baselines, omitting competitive tree models, or evaluating competing methods under mismatched lookback windows all produce the same outcome for production engineers—the remarkable gains reported in the paper vanish under production conditions.

When reading such literature, engineers can protect their architectures by routinely asking three questions:

1. Did the authors run stripped-down ablations and minimalist baselines? If the complex language model backbone is removed and replaced with a single linear layer or shallow network, does predictive accuracy actually degrade?
2. Are evaluation settings strictly consistent? Are all competing methods benchmarked across identical train/val/test splits, matching lookback windows, and equivalent feature engineering?
3. Are code and data publicly verifiable? Is the core implementation open-sourced? Can the evaluation workflow be independently reproduced on open benchmark repositories (such as the Monash Time Series Forecasting Archive [30], GIFT-Eval [27], or TIME [21])? If gains are reported on private datasets, are baseline comparisons and schemas clearly articulated?

5.3 From the Ivory Tower to the Production Pipeline: The Real Engineering Bottom Line #

Academia can explore the theoretical boundaries of frontier hypotheses on curated benchmark datasets. Eventually, however, any technical solution must leave the ivory tower and withstand the unforgiving realities of physical systems and multi-million-dollar transaction streams.

In industrial production, architectures must not only optimize for accuracy metrics, but also account for inference latency, infrastructure costs, operational resilience across cold and warm starts, and tangible business profit and loss (P&L). Academic margins propped up by weak baselines collapse rapidly under thousands of live production queries per second. Genuine technical confidence never requires obscuring operational boundaries.

In this regard, Tsinghua’s Sundial [20] offers an exemplary standard of scholarly transparency: the authors explicitly excluded benchmark datasets during pretraining to prevent data leakage, released weights and code under a permissive Apache-2.0 license, and candidly documented the model’s limitations in the appendix (including its univariate design without covariate support, unproven guarantees on high-frequency series due to low-frequency pretraining dominance, and the risk of hallucinated trajectories).

Laying assumptions, operating boundaries, and potential shortcomings out in plain sight represents the true rigor of academic exploration—and provides the trustworthy foundation that industrial engineers rely upon when making architectural decisions. When teams refuse to chase ungrounded hype and refuse to conceal limitations, engineering progress advances on solid ground.

6. Industrial Decision Matrix and Guidelines #

Having examined the boundaries of scholarly inquiry and methodological rigor, our journey inevitably returns to the engineering trenches—a world governed by hard operational constraints, latency SLAs, and compute balance sheets.

As an author, I must candidly emphasize that the decision matrix and stepwise workflow presented below are neither prescriptive dogma nor an infallible industry standard. Rather, they represent a structured conceptual framework synthesized from the multi-tiered thought experiments in Chapters 3 and 4, paired with hard-won lessons from hands-on demand forecasting deployments.

In production reality, every enterprise operates under distinct data maturity, legacy codebases, hardware budgets, and stakeholder explainability requirements. The profiles and flowcharts that follow are intended simply as a pragmatic reference point to stimulate thoughtful peer discussion, helping engineering teams weigh realistic trade-offs before committing infrastructure resources.

In enterprise systems architecture, the highest engineering virtue remains delivering dependable business accuracy at the lowest sustainable system complexity and cost. The table below summarizes the operational profiles of the five mainstream paradigms:

Stepwise Engineering Selection Workflow #

When architecting a production forecasting system, model selection should never be a one-size-fits-all bet; instead, teams should follow a stepwise engineering decision workflow:

1. Step 0: Build the Data Observability Foundation First—Before tuning any model, impute missing values, align asynchronous timestamps, unbias stockout-censored sales, and curate holiday/promotion calendars;
2. Step 1: Route by Historical Depth (Cold-Start vs. Mature Series)—Send new products, new stores, and newly installed sensors with minimal history into the Cold-Start Track, while routing established series with multiple seasonal cycles into the Core Production Track;
3. Step 2: Select by Compute Budget and Attribution Requirements—On the Cold-Start Track, deploy native foundation models (Chronos-2, TimesFM 3.0) in zero-shot mode when GPU budget is available, or fall back to Look-alike clustering + FSM on strict CPU budgets. On the Core Production Track, use partitioned GBDTs for high-volume tabular panels, classical statistical models for highly seasonal single series, and dedicated deep models for continuous physical telemetry;
4. Step 3: Run Zero-Shot Shadow Challengers—Alongside mature production pipelines, run Chronos-2 or TimesFM 3.0 in silent shadow mode to continuously benchmark zero-shot generalization ceilings at minimal engineering cost.

The flowchart below maps this end-to-end decision process:

Pragmatic Engineering Principles #

1. Always spend the first dollar on data governance, not model upgrades: Without clean observability and stockout unbiasing, larger models merely memorize historical distortions.
2. Benchmark every complex architecture against a tuned GBDT baseline: Before provisioning GPU clusters for a foundation model, run a fair comparison against a well-featured LightGBM pipeline.
3. Run a dual-track strategy for cold starts (Native Foundation Models + Look-alike Fallback): Use Chronos-2 or TimesFM 3.0 for zero-shot cold-start extrapolation, while retaining Look-alike proxy rules as a deterministic fallback when GPU budgets or strict explainability constraints apply.
4. Use native foundation models as low-cost “shadow challengers”: Calling a zero-shot foundation model alongside an existing GBDT pipeline takes only a few lines of code and provides an ongoing reference check on potential accuracy headroom.

Key Concept Clarifier:

• Challenger Baseline: Borrowed from the Champion-Challenger deployment pattern in financial risk and industrial MLOps. The model actively serving live production decisions is the Champion (e.g., a partitioned LightGBM pipeline), while a newer candidate (such as Chronos-2 or TimesFM 3.0) runs silently in parallel on the same live inputs (Shadow Run) as the Challenger—letting objective production metrics decide whether a migration is justified.

7. Conclusion and Outlook: What Lies Ahead for Time Series Foundation Models #

With the technical and architectural evidence laid out, we can directly answer the question posed in the title: Should we build a time series foundation model inside a thriving forest of GBDT trees?

7.1 Conclusion: Do Not Clear-Cut the Forest—Build the Foundation in Its Blind Spots #

The answer depends entirely on where an engineering team plans to place that foundation and what problem it is meant to solve:

• If the goal is to clear-cut the existing GBDT forest and statistical pipelines to replace them entirely—then the answer is no, and the unit economics will not pencil out in the foreseeable future. As shown across our earlier analyses of Prophet in retail demand forecasting, tree models, and graph neural networks, data observability (stockout unbiasing, anomaly cleaning) still sets the accuracy ceiling across the mature product lines that generate the vast majority of volume. When handling wide tabular business features, nonlinear price-tier elasticity, strict monotonic constraints, SHAP attribution, and multi-billion-row CPU batch throughput, a partitioned LightGBM or XGBoost forest remains the decisive mainstream choice. Grafting multi-billion-parameter language models onto pure numerical series is even harder to justify on an engineering balance sheet.
• If the goal is to address the local blind spots where decision trees struggle most—then not only is it worth introducing, but native time series foundation models led by Chronos-2, TimesFM 3.0, and Sundial already have solid foundations for targeted production pilots in cold-start and shadow-challenger roles today. In cold-start product and store launches, short-history promotional extrapolation, cross-series group attention acting as “dynamic soft proxies,” and low-cost zero-shot shadow challengers alongside main pipelines, a compact native foundation model eliminates brittle manual state machines and early-stage feature engineering overhead (before commercial production rollout, teams should also verify open-source weight licenses—for instance, preferring Apache-2.0 models such as Chronos-2 or Sundial while noting the non-commercial restrictions on TimesFM 3.0 PyTorch weights).

Practitioners therefore need neither mystify foundation models nor cling dogmatically to a single tool. The most resilient production architecture today treats data governance as the soil, partitioned tree models as the mainstream forest (carrying mature volume), and native time series foundation models as a specialized complement (tackling cold-start and zero-shot frontiers), with a smooth handoff across the product lifecycle. The visionary height of the foundation model illuminates the blind spots deep within the forest; while upon the dense, grounded roots of the forest, the tower stands firm rather than collapsing as an ephemeral castle in the air.

7.2 Forward Outlook: Three Pragmatic Trajectories Over the Next Two to Three Years #

Looking ahead from late 2026, as the early hype around forcing language models onto numerical time series fades, industrial adoption is shifting from chasing a single monolithic architecture toward pragmatic division of labor driven by business unit economics and operational returns. Synthesizing recent research with real-world enterprise balance sheets points to three practical developments over the next two to three years:

1. Model Synergy: Using Time Series Foundations as Zero-Shot Feature Extractors for GBDT Wide Tables
   • Business driver: In core commercial domains such as retail, e-commerce, and logistics, enterprise data warehouses store massive high-cardinality static attributes and domain rules (such as multi-level merchandise hierarchies, store trade-area demographics, and cross-price tiers), while downstream merchandising and risk teams rely heavily on the monotonic constraints and attribution explainability of tree models. No enterprise will rip out a battle-tested GBDT wide-table pipeline that carries core revenue just to adopt a new model family—especially when current native models like Chronos-2 and TimesFM 3.0 remain less direct than decision trees at digesting ultra-wide static tabular features.
   • Trajectory and expected benefit: Rather than waiting for an all-in-one architecture that handles both wide tables and continuous sequences equally well, production teams are adopting feature-level fusion: using a compact time series foundation model to extract zero-shot quantile priors or sequence embeddings for cold-start and short-history items, and feeding those outputs as a few new “sequence prior columns” into downstream GBDT wide tables (meanwhile, tabular foundation models such as TabPFN-TS are also showing promise for tuning-free prior extraction on narrow cold-start tables, enriching the feature toolkit around the GBDT forest). This approach preserves the GBDT forest’s control over complex business rules and low-cost CPU batch throughput while borrowing the foundation model’s cross-series cold-start generalization. Engineering teams can lift baseline accuracy on new launches and long-tail SKUs through a low-risk incremental feature addition without rebuilding existing production pipelines.
2. Output Evolution: From Point Mean Forecasts to Probabilistic Intervals Driving Inventory and Working Capital
   • Business driver: On a real supply chain balance sheet, forecasting errors are never symmetric—overstocking fresh daily perishables leads directly to spoilage write-offs, whereas understocking high-margin durable goods causes stockouts and customer churn. Commercial decision systems do not need a single point expectation that shaves 0.01 off test-set MSE; they need quantified uncertainty bounds. Moreover, as the TIME benchmark [21] demonstrated, models that output only smooth point expectations often degenerate into flat mean lines during volatile periods, offering little operational value.
   • Trajectory and expected benefit: As native quantile heads and generative flow matching (such as Sundial [20]) mature, the standard deliverable of time series foundation models is shifting from single point estimates to multi-horizon quantile intervals (e.g., P10/P50/P90) and joint probabilistic trajectories. The tangible commercial value is direct: downstream replenishment engines can select optimal service-level quantiles tailored to each category’s gross margin and spoilage rate (for example, stocking high-margin core items at P85 while capping high-spoilage perishables at P40), simultaneously reducing stockout rates and dead-stock write-offs without increasing total working capital tied up in inventory—bridging the last mile from statistical metrics to business P&L.
3. Infrastructure and Agentic Division of Labor: Forecasting Operators in the Warehouse, Language Models in the Business Orchestration Layer
   • Business driver: On one hand, most traditional retail, manufacturing, and mid-market BI teams lack dedicated deep-learning infrastructure engineers and cannot justify the ongoing cost of standalone GPU inference clusters. On the other hand, daily commercial operations are constantly disrupted by unstructured events (severe weather alerts, port closure notices, competitor flash sales, cross-departmental promotional changes) that cannot be fed directly into numerical models—yet paying for a multi-billion-parameter LLM to compute floating-point trajectories across millions of SKUs makes no financial sense.
   • Trajectory and expected benefit: These two commercial pressures are driving a clean architectural decoupling. At the infrastructure layer, compact 10M–500M native time series foundation models are sinking rapidly into cloud data warehouses and lakehouses as built-in SQL operators (such as BigQuery AI.FORECAST). At the application layer, large language models (LLMs) return to where they excel—serving as the semantic understanding and tool-orchestration hub inside enterprise Agent Systems, as explored in Do Enterprises Really Need AI Agents? When to Let Models Decide the Next Step and When AI Agents Begin Modifying Business State: The Need for a Runtime Framework Beyond the Model. When an external disruption or promotional shift occurs, the LLM extracts structured intervention parameters from unstructured text, invokes the underlying GBDT forest and time series foundation operators to run multi-scenario counterfactual (“what-if”) simulations, and translates the resulting predictive distributions into auditable replenishment orders inside a governed runtime framework. Let specialized models handle what they do best: LLMs no longer attempt to crunch floating-point sales figures (an approach that is both expensive and error-prone), but act instead as strategic orchestrators of business intent—delegating high-concurrency numerical computations to underlying GBDT partitions and warehouse time-series primitives. In-warehouse operators let business analysts invoke zero-shot probabilistic forecasts with a single line of standard SQL at a fraction of the infrastructure TCO, while the agentic division of labor—LLMs for semantic orchestration, specialized models for numerical distributions—can compress cross-functional contingency planning from days of manual alignment down to minutes.

Every technological wave passes through the crucible from conceptual euphoria to engineering pragmatism. Whether confronting dazzling new architectures in top-tier conference papers or billions of transaction rows in production pipelines, guarding the baseline of data quality, accounting honestly for hardware and maintenance costs, and remaining rigorous with empirical evidence will always be the most reliable anchor for engineers and researchers alike.

Further Reading #

The following papers provide essential context on the evolution of time series forecasting and modern evaluation benchmarks:

1. Tan et al. (NeurIPS 2024) [4]: Systematic ablation study demonstrating that frozen language model backbones offer no accuracy advantage over lightweight layers on numerical time series; essential reading on cross-modal transfer;
2. Makridakis et al. (IJF 2022) M5 Competition Summary [24]: Comprehensive evaluation across 42,000+ hierarchical retail series establishing the practical effectiveness of LightGBM and feature engineering in retail forecasting;
3. Qiu et al. (VLDB 2024) TFB Benchmark [25]: Fair evaluation of 35 methods across 8,000+ series demonstrating the solid resilience of simple linear baselines on noisy physical datasets;
4. Qiao et al. (ICML 2026) TIME Benchmark [21]: Critical audit of data quality flaws and artificial horizons in legacy benchmarks, introducing a strictly leak-free zero-shot evaluation suite;
5. Ansari et al. (2025) Chronos-2 [8]: Landmark native time series foundation model replacing discrete binning with continuous patch projections and native exogenous covariate support;
6. Jain & Sen (2026) TimesFM-3 [9]: Google’s native multivariate time series foundation model featuring alternating temporal and variate attention with flexible covariate channels;
7. Taylor & Letham (2018) Prophet [10]: Widely deployed additive decomposition model offering intuitive holiday priors and component-level business interpretability;
8. Lim et al. (IJF 2021) TFT [13]: Temporal Fusion Transformer combining local sequence processing, multi-horizon attention, and dynamic variable selection;
9. Liu et al. (ICML 2025) Sundial [20]: Tsinghua University’s 128M generative flow-matching foundation model exploring non-parametric continuous trajectory sampling.

References #

1. T. Zhou, P. Niu, X. Wang, L. Sun, and R. Jin, “One fits all: Power general time series analysis by pretrained LM,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 36, pp. 43322-43355, 2023. arXiv:2302.11939
2. M. Jin et al., “Time-LLM: Time series forecasting by reprogramming large language models,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2024. arXiv:2310.01728
3. P. Liu et al., “CALF: Aligning LLMs for time series forecasting via cross-modal fine-tuning,” arXiv preprint arXiv:2403.07300, 2024. arXiv:2403.07300
4. M. Tan, M. A. Merrill, V. Gupta, T. Althoff, and T. Hartvigsen, “Are language models actually useful for time series forecasting?” in Adv. Neural Inf. Process. Syst. (NeurIPS), 2024. arXiv:2406.16964
5. A. F. Ansari et al., “Chronos: Learning the language of time series,” Trans. Mach. Learn. Res. (TMLR), 2024. arXiv:2403.07815
6. A. Das, W. Kong, R. Sen, and Y. Zhou, “A decoder-only foundation model for time-series forecasting,” in Proc. Int. Conf. Mach. Learn. (ICML), 2024. arXiv:2310.10688
7. G. Woo, C. Liu, A. Kumar, C. Xiong, S. Savarese, and D. Sahoo, “Unified training of universal time series forecasting transformers,” in Proc. Int. Conf. Mach. Learn. (ICML), 2024. arXiv:2402.02592
8. A. F. Ansari, O. Shchur, J. Küken, et al., “Chronos-2: From univariate to universal forecasting,” arXiv preprint arXiv:2510.15821, 2025. arXiv:2510.15821
9. A. Jain and R. Sen, “TimesFM-3: A zero-shot foundation model for multivariate forecasting,” Google Research Blog, 2026. [Online]. Available: https://research.google/blog/timesfm-3-a-zero-shot-foundation-model-for-multivariate-forecasting/
10. S. J. Taylor and B. Letham, “Forecasting at scale,” The American Statistician, vol. 72, no. 1, pp. 37-45, 2018. DOI:10.1080/00031305.2017.1380080
11. G. Ke et al., “LightGBM: A highly efficient gradient boosting decision tree,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 30, pp. 3146-3154, 2017. NeurIPS Proceedings
12. T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” in Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., pp. 785-794, 2016. DOI:10.1145/2939672.2939785
13. B. Lim, S. O. Arik, N. Loeff, and T. Pfister, “Temporal Fusion Transformers for interpretable multi-horizon time series forecasting,” Int. J. Forecast., vol. 37, no. 4, pp. 1748-1764, 2021. arXiv:1912.09363
14. A. Zeng, M. Chen, L. Zhang, and Q. Xu, “Are transformers effective for time series forecasting?” in Proc. AAAI Conf. Artif. Intell., vol. 37, no. 9, pp. 11121-11128, 2023. arXiv:2205.13504
15. S.-A. Chen, C.-L. Li, N. Yoder, S. O. Arik, and T. Pfister, “TSMixer: An all-MLP architecture for time series forecasting,” Trans. Mach. Learn. Res. (TMLR), 2023. arXiv:2303.06053
16. A. Das et al., “Long-term forecasting with TiDE: Time-series dense encoder,” Trans. Mach. Learn. Res. (TMLR), 2023. arXiv:2304.08424
17. Y. Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam, “A time series is worth 64 words: Long-term forecasting with transformers,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2023. arXiv:2211.14730
18. Y. Liu et al., “iTransformer: Inverted transformers are effective for time series forecasting,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2024. arXiv:2310.06625
19. Y. Liu, H. Zhang, C. Li, X. Huang, J. Wang, and M. Long, “Timer: Generative pre-trained transformers are large time series models,” in Proc. 41st Int. Conf. Mach. Learn. (ICML), 2024. arXiv:2402.02368
20. Y. Liu, G. Qin, Z. Shi, Z. Chen, C. Yang, X. Huang, J. Wang, and M. Long, “Sundial: A family of highly capable time series foundation models,” in Proc. 42nd Int. Conf. Mach. Learn. (ICML), PMLR 267, 2025, pp. 39295-39317. arXiv:2502.00816. Code: https://github.com/thuml/Sundial
21. Z. Qiao, S. Pan, A. Wang, V. Zhukova, Y. Liu, X. Jiang, Q. Wen, M. Long, M. Jin, and C. Liu, “It’s TIME: Towards the next generation of time series forecasting benchmarks,” in Proc. 43rd Int. Conf. Mach. Learn. (ICML), PMLR 306, 2026. arXiv:2602.12147
22. H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long sequence time-series forecasting,” in Proc. AAAI Conf. Artif. Intell. (AAAI), vol. 35, no. 12, pp. 11106-11115, 2021. Data: https://github.com/zhouhaoyi/ETDataset
23. T. Kim, J. Kim, Y. Tae, C. Park, J.-H. Choi, and J. Choo, “Reversible instance normalization for accurate time-series forecasting against distribution shift,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2022. OpenReview
24. S. Makridakis, E. Spiliotis, and V. Assimakopoulos, “The M5 competition: Background, organization, and implementation,” Int. J. Forecast., vol. 38, no. 4, pp. 1325-1336, 2022. DOI:10.1016/j.ijforecast.2021.11.013
25. X. Qiu et al., “TFB: Towards comprehensive and fair benchmarking of time series forecasting methods,” Proc. VLDB Endow., vol. 17, no. 9, pp. 2363-2377, 2024. arXiv:2403.20150
26. O. Shchur, A. Turkmani, Y. Zhang, B. N. Oreshkin, and A. F. Ansari, “fev-bench: A unified framework and benchmark for time series forecasting evaluation,” arXiv preprint arXiv:2509.26468, 2025. arXiv:2509.26468
27. G. Woo, C. Liu, A. Kumar, C. Xiong, S. Savarese, and D. Sahoo, “GIFT-Eval: A benchmark for general time series forecasting evaluation,” arXiv preprint arXiv:2406.06688, 2024. arXiv:2406.06688
28. J. D. Croston, “Forecasting and stock control for intermittent demands,” J. Oper. Res. Soc., vol. 23, no. 3, pp. 289-303, 1972. DOI:10.1057/jors.1972.50
29. H. Hersbach et al., “The ERA5 global reanalysis,” Q. J. R. Meteorol. Soc., vol. 146, no. 730, pp. 1999-2049, 2020. DOI:10.1002/qj.3803. Data: Copernicus Climate Data Store (CDS).
30. R. Godahewa, C. Bergmeir, G. I. Webb, R. J. Hyndman, and P. Montero-Manso, “Monash Time Series Econometrics Forecasting Archive,” in Adv. Neural Inf. Process. Syst. (NeurIPS) Track on Datasets and Benchmarks, 2021. arXiv:2105.06643. Data: https://forecastingdata.org/