Time Series Foundation Models - Google TimesFM (Part 1): Model, Algorithms, and Engineering Practice

In our previous guide, Should We Build a ‘Tower of Babel’ in a Forest of Trees? Time Series Foundation Models vs. GBDT in Practice, we distinguished “repurposed language models” from time-series-native foundation models (TSFMs). Compact architectures pre-trained directly on continuous numerical sequences are one important option for enterprise evaluation, alongside statistical and supervised machine-learning systems.

Concept Primer: What is a Time Series Foundation Model, and what is Zero-Shot Forecasting? A time-series foundation model (TSFM) is a high-capacity deep neural network pre-trained across hundreds of billions of continuous observation points spanning diverse industries and sampling frequencies (from sub-second telemetry to monthly accounting aggregates). Its primary operational appeal lies in zero-shot forecasting: when presented with an unseen warehouse SKU, newly opened store, or unfamiliar equipment, the model requires no local fine-tuning or parameter retraining. Given only raw historical observations, it projects future demand directly using the geometric wave shapes and periodic priors acquired during pre-training.

Following that article, readers raised a practical question: open any mainstream Hugging Face time-series benchmark—such as GIFT-Eval [1], fev-bench [2], or ICML 2026’s TIME benchmark [3]—and you are greeted by dozens of competing models: Chronos-2, TimesFM, Moirai, Sundial, Timer, TiRex, and more. Production engineers have neither the time to read every paper nor the compute to benchmark every published checkpoint on live pipelines. Why single out Google’s TimesFM for an end-to-end dissection from mathematical equations to enterprise SQL deployment?

There are two straightforward and compelling reasons:

First, while model names are legion, underlying architectures have converged—“dissecting one sparrow reveals the anatomy of the flock.” Stripping away marketing jargon, with the exception of early attempts to discretize continuous values into vocabulary bins (such as the first generation of Chronos [4]), leading native foundation models share a common architectural foundation: local instance normalization, continuous patching (segmenting 1D waveforms into sequential tokens), causal Transformer attention, multi-step point and quantile output heads, and mixed real-world and synthetic pre-training. Rather than skimming superficial parameter tables across twenty models, thoroughly deconstructing a battle-tested industrial architecture—from input patch projection and mask alignment to quantile loss formulation—equips you to immediately recognize which components other models modify.

Second, TimesFM is one of the few research subjects available through both open Python checkpoints and managed cloud interfaces. Many benchmark-topping models stall after publication as static checkpoint snapshots. TimesFM instead evolved from its ICML 2024 debut [5] through versions 2.0 and 2.5 to version 3.0 in August 2026 [6], leaving architecture, code, and deployment contracts that engineers can compare. A team can pin a Python checkpoint revision for a reproducible baseline or evaluate managed paths such as BigQuery AI.FORECAST and Vertex AI [7]. Those paths have different input contracts, licenses, and costs; sharing the TimesFM name does not make them the same runtime.

Core Architecture and Decisions at a Glance

• Architectural Evolution: TimesFM 1.0–2.5 use a decoder-only causal architecture with input patches of \(p=32\) and output patches of \(h=128\) to reduce the number of long-horizon rollouts. TimesFM 3.0 retains 32-step input patches, changes the output patch to 64, and uses Contiguous Patch Masking (CPM) to fill the requested horizon in one forward pass rather than rolling forward from the final history token [6].
• Arbitrary-Length Generalization: Using random prefix masking (RPM) during pre-training, the model learns to forecast from any remainder offset within a patch, seamlessly accommodating truncated cold-start sequences of only a few days without retraining.
• Covariates and Deployment: Pure univariate models remain blind to promotional spikes and holiday closures. Real-world systems must integrate business schedules via XReg (contextual ridge regression residual decomposition) or TimesFM 3.0’s native alternating cross-variable attention. For deployment, BigQuery AI.FORECAST offers serverless zero-ops execution, while teams self-hosting weights must observe the licensing boundary between TimesFM 2.5 (Apache-2.0) and TimesFM 3.0 (non-commercial open weights).
• Reproducible Experiment Interface: The companion benchmark pins a specific revision of google/timesfm-3.0-pytorch and runs separate univariate and known-covariate Python configurations. BigQuery remains a deployment option, but it is not the source of the current experiment results.

1. Evolution and Business Drivers #

Before examining equations, tracing the version timeline illustrates why Google Research re-engineered TimesFM four times. Each release addressed an explicit obstacle encountered in production:

The engineering drivers behind this evolution are transparent:

• TimesFM 1.0 (Validating Asymmetric Patching): Established the Patch (p=32 -> h=128) paradigm, 20-layer stacked decoder, and joint point/quantile heads, demonstrating that continuous patch pre-training could match supervised baselines on univariate benchmarks zero-shot.
• TimesFM 2.0 (Expanded Capacity and XReg): Increased the parameter count to 500M, extended context to 2,048 steps, and introduced the XReg interface for external signals such as holidays and promotions.
• TimesFM 2.5 (Smaller Backbone and 16k Context): Moved to a 200M backbone, extended context to 16,384 steps, removed the explicit frequency argument, and provided an optional 30M continuous quantile head.
• TimesFM 3.0 (Native Multivariate and In-Line Covariates): Pre-trained on over 1 trillion points at 330M parameters [6], it interleaves causal temporal attention with full variate attention. Lookahead tokens expose future-known covariates, while CPM generates the full requested horizon in one pass. Public PyTorch weights use a non-commercial license; commercial use is available under Google Cloud terms, or teams can self-host the Apache-2.0 version 2.5 weights [7][9].

2. Patching and Decoding by Model Generation #

Why do large language models generate text point-by-point (token-by-token autoregression), while time-series foundation models avoid scalar autoregression?

Two structural constraints explain the divergence:

1. Scalar Observations Lack Semantic Density: A word token carries rich standalone semantics. In contrast, a single continuous numerical value (such as \(y_t = 102.0\)) stripped of local slope, curvature, and volatility carries minimal geometric meaning. Treating each scalar in a length \(L=2048\) series as an individual token inflates self-attention compute to \(O(L^2)\).
2. More Opportunities for Autoregressive Error Accumulation: Retail systems rarely forecast a single step ahead. They require 28-day replenishment horizons or 720-hour utility schedules. Predicting one step \(\hat{y}_{L+1}\), appending it to the context, and rolling forward dozens of iterations increases latency and repeatedly uses prior predictions as inputs, allowing early errors to propagate and potentially grow.

To resolve both issues simultaneously, Das et al. [5] engineered an asymmetric patching decoder-only architecture. The following three stages describe TimesFM 1.0–2.5. TimesFM 3.0 retains 32-step input patches but replaces this decoding path with CPM, which Section 2.4 describes separately:

• Stage 1 (Input Representation Layer): Continuous patching with stride \(p=32\), RevIN scale stripping, and feature projection;
• Stage 2 (Causal Decoder Backbone Layer): 20-layer Transformer decoder backbone aggregating the full historical receptive field;
• Stage 3 (Output Projection Layer): Direct single-pass projection to horizon \(h=128\), producing point forecasts and P10–P90 risk corridors.

The mathematical mechanics and tensor transformations of each stage are detailed below:

2.1 Input Patching and RevIN #

Following standard column-vector convention, given historical observations \(\mathbf{y}_{1:L} = (y_1, \dots, y_L)^\top \in \mathbb{R}^L\):

1. Continuous Patching: The series is segmented with fixed stride \(p=32\) into \(N = \lceil L / p \rceil\) non-overlapping patch column vectors:

If sequence length \(L\) is not an exact multiple of 32, the leading edge is left-padded with zeros alongside a binary indicator mask \(\mathbf{m}_j \in \{0, 1\}^p\) (\(m_{j,k}=1\) denotes padded or masked positions; \(m_{j,k}=0\) denotes valid historical observations).

2. Reversible Instance Normalization (RevIN): Strips local instance mean \(\mu\) and standard deviation \(\sigma\) prior to attention:

where \(\mathbf{1} = (1, \dots, 1)^\top \in \mathbb{R}^p\) and \(\epsilon = 10^{-5}\) prevents division-by-zero on flat series. This operation mirrors RevIN [8], enabling identical model weights to generalize across micro-sales and bulk electrical loads.

3. Input Residual Block: Normalized patches and binary masks concatenate into feature vector \(\mathbf{u}_j \in \mathbb{R}^{2p}\), projected into Transformer model dimension \(d_{\text{model}} = 1280\) with absolute temporal positional encodings:

2.2 Causal Decoder Backbone #

Token sequence \((\mathbf{t}_1, \dots, \mathbf{t}_N)\) enters a Transformer decoder backbone. The equations below use the original 200M architecture as the concrete example: 20 decoder layers, 16 attention heads per layer, and \(d_{\text{model}}=1280\). Layer count and capacity differ across releases.

In TimesFM 1.0 through 2.5, the backbone enforces strictly causal self-attention via lower-triangular masks: token \(j\) attends exclusively to historical tokens \(\{\mathbf{t}_1, \dots, \mathbf{t}_j\}\), preventing any future leakage. This guarantees two key architectural strengths:

• Parallel Training: Similar to autoregressive language modeling, an entire length-\(N\) sequence evaluates losses across all historical cutoffs in a single forward pass.
• Context Length Flexibility: A checkpoint accepts different history lengths, from short sequences up to its configured context limit; that limit varies by release.

2.3 Long-Horizon Output Projection #

In the original 200M architecture, the final token’s top-level hidden state \(\mathbf{h}_N^{(20)} \in \mathbb{R}^{d_{\text{model}}}\) maps through an Output Residual Block directly to future horizon \(h=128\):

In the older output interface, this yields one point forecast alongside nine quantiles \(\mathcal{Q} = \{0.1, \dots, 0.9\}\):

• Point Forecast: The exact statistic depends on the version and API. BigQuery defines forecast_value as the P50 conditional median, not the conditional mean [7].
• P90 Quantile: Under the model’s conditional distribution, demand has a 10% probability of exceeding this value. It can inform an inventory decision, but it is not automatically a safety-stock bound or a 90% service-level guarantee.
• P10 Quantile: Under the model’s conditional distribution, demand has a 10% probability of falling below this value. It can inform downside-risk analysis, but it is not a direct clearance or inventory threshold.
• P10–P90 Prediction-Interval Width: Describes dispersion in the model output; calibration still requires evaluation across many series and forecast windows.

Why the Asymmetric Design (\(p=32 \to h=128\))?

Ablation experiments by Das et al. [5] confirmed the engineering necessity of this ratio:

• Input Patch Must Remain Compact (\(p=32\)): Excessively wide input strides (e.g., \(p=128\)) over-smooth inflection points and cannot accommodate cold-start sequences of 14–30 days.
• Output Horizon Must Remain Substantial (\(h=128\)): Shorter output projections (\(h=32\) or \(h=8\)) force 4 to 16 autoregressive rollout steps to cover standard replenishment cycles. A 128-step output reduces rollout latency and the opportunity for cumulative drift, but does not eliminate long-horizon forecast error.

2.4 TimesFM 3.0: CPM Non-Autoregressive Decoding #

TimesFM 3.0 does not simply extend the 128-step output head from versions 1.0–2.5 across a multivariate grid. Its official configuration uses 32-step input patches, 64-step output patches, and Contiguous Patch Masking (CPM) [6]:

1. The model appends placeholder tokens that cover the requested forecast horizon.
2. Target variables and past-only covariates remain masked in the future, while future-known covariates remain visible.
3. Lookahead construction lets a current covariate token carry information from planned future patches.
4. Alternating causal temporal and full variate attention propagates this information, and the model fills all future placeholders in one forward pass with nine quantiles from P10 through P90.

The phrase “single forward pass” therefore has a different meaning in version 3.0. It does not project 128 values from the final history token or append predicted patches for another rollout. CPM decodes the requested horizon through future placeholder tokens.

2.5 Exogenous Covariate Integration #

Having established the foundational univariate pipeline, production systems must integrate real-world commercial signals:

where \(g(t)\) represents underlying trend, \(s(t)\) seasonality, and \(h(t)\) promotional and holiday shocks. Exogenous variables are strictly classified by temporal availability:

1. Future-Known Business Covariates (defined over \(L + H\)): Promotional pricing calendar, scheduled store closures (is_open = 0), and statutory holiday indicators (is_holiday).
2. Observed Historical Covariates (defined over \(L\), forecast-dependent in future): Ambient temperatures \(\text{Temp}_t\), where future periods require meteorological forecasts or seasonal norms.

TimesFM addresses covariate integration across two generational paradigms:

• Paradigm A (TimesFM 2.0 / 2.5: Decoupled XReg Workflow): XReg denotes Exogenous Regressors. Retaining the univariate purity of the Transformer backbone, it pairs the model with an external ridge regression solved over history in three steps:
  1. Shock Extraction: Fits local ridge regression over historical lookback \(1:L\): \[ \hat{\boldsymbol{\beta}} = \left(\mathbf{X}_{1:L}^\top \mathbf{X}_{1:L} + \lambda \mathbf{I}\right)^{-1} \mathbf{X}_{1:L}^\top \mathbf{y}_{1:L} \] to isolate promotional, closure, and temperature coefficients;
  2. Residual Forecasting: Strips external pulses to obtain stationary residuals \(\mathbf{r} = \mathbf{y} - \mathbf{X}\hat{\boldsymbol{\beta}}\), forecast zero-shot by the pure univariate TimesFM backbone;
  3. Signal Reconstruction: Linearly superimposes future plans \(\mathbf{X}_{\text{future}}\hat{\boldsymbol{\beta}}\) onto forecasted residuals \(\hat{\mathbf{r}}\) to reconstruct the final prediction.
• Paradigm B (TimesFM 3.0: Native Multivariate Grid & Alternating Attention): Upgrades inputs into a multi-channel tensor grid \([B,\, C,\, N,\, d_{\text{model}}]\) (where \(C\) is the number of channels) [6]. The backbone alternates between:
  1. Causal Temporal Attention: Propagating signals unidirectionally along time \(N\) within each variable channel, enforcing strict causality;
  2. Cross-Channel Attention: Bidirectionally attending across all \(C\) channels within each patch \(j\), learning non-linear interactive effects (such as heatwaves amplified by price discounts) end-to-end.

2.6 End-to-End Tensor Lifecycle #

We trace an instance with \(L=50\) predicting \(H=28\) days under promotional discounts, store closures, and daily temperatures.

1. Business Inputs and Covariates #

• Target Sales \(\mathbf{y}_{1:50} \in \mathbb{R}^{50}\)
• Covariates \(\mathbf{X} \in \mathbb{R}^{78 \times 3}\): discount, closure indicator, and temperature across \(L+H = 78\) days.
• Forecast Horizon: Future \(H=28\) days point forecast and P10–P90 corridor.

2. XReg Tensor Lifecycle #

1. Ridge Regression Shock Removal: Solve \(\hat{\boldsymbol{\beta}} \in \mathbb{R}^3\) and extract stationary residuals \(\mathbf{r}_{1:50} = \mathbf{y}_{1:50} - \mathbf{X}_{1:50}\hat{\boldsymbol{\beta}}\).
2. Patching and Mask Assembly (\([1, 50] \to [1, 2, 64]\)): With stride \(p=32\), \(N = \lceil 50/32 \rceil = 2\). Left-pad 14 zeros on patch 1 with mask \(\mathbf{m}_1\); patch 2 is fully observed. Yields feature tensor \([1, 2, 64]\).
3. Input Residual Block (\([1, 2, 64] \to [1, 2, 1280]\)): Projects into initial tokens \(\mathbf{H}^{(0)} \in \mathbb{R}^{2 \times 1280}\).
4. 20 Causal Decoder Blocks: Evaluates causal attention, condensing context into final token hidden state \(\mathbf{h}_2^{(20)} \in \mathbb{R}^{1280}\).
5. Output Projection and Signal Reconstruction (\([1, 1280] \to [1, 128, 10] \to [1, 28, 10]\)): \(\operatorname{ResBlock}_{\text{out}}\) produces 128-step point and quantile predictions. After denormalization, add future plans \(\mathbf{X}_{51:78}\hat{\boldsymbol{\beta}}\) and slice the first 28 steps.

3. TimesFM 3.0 Grid and Placeholder Tokens #

TimesFM 3.0 integrates covariates inside the network, but future-known covariates are not merely stacked into a history-only tensor of the same shape:

• Historical patches: Sales, discounts, open indicators, and temperatures are grouped into 32-step patches on a variate-by-time token grid.
• Future placeholders: A 28-day request appends masked placeholders for sales and past-only variables.
• Lookahead covariates: Planned discounts and trading calendars remain visible and carry future-patch information in their covariate tokens.
• Alternating attention and decoding: Temporal attention remains causal, variate attention connects channels at a time position, and CPM produces nine quantiles for the entire 28-day horizon in one pass.

Exact shapes such as [1, 4, 2, 64] -> [1, 4, 2, 1280] -> [1, 28, 10] are intentionally omitted because the token count also depends on future placeholders, lookahead construction, batching, and the serving API. The stable interface is the distinction among targets, past-only covariates, and past-future covariates.

3. Pre-Training Algorithms #

3.1 Random Prefix Masking #

Training solely on multiples of 32 induces catastrophic phase misalignment when encountering irregular lengths (e.g., cold-start items with 14 days of history).

TimesFM introduces Random Prefix Masking (RPM):

1. Sample an offset \(r \sim \operatorname{Uniform}\{0, \dots, p-1\}\);
2. Mask the initial \(r\) points of the first patch (\(m_{1,1} = \dots = m_{1,r} = 1\));
3. The model learns to project future horizons from partial prefix contexts.

During inference, irregular sequences are automatically left-padded, and the final token smoothly projects from step \(L+1\).

3.2 Composite Quantile Loss #

TimesFM optimizes joint point MSE and multi-horizon Pinball loss [5]:

where Pinball loss penalizes residuals asymmetrically:

For P90, under-forecasting incurs nine times the penalty of an equally sized over-forecast. This asymmetric objective targets the conditional 90th percentile, but it does not by itself guarantee out-of-sample calibration or define a safety-stock policy.

3.3 Real and Synthetic Pre-Training #

The original TimesFM 1.0 paper reports a pre-training mixture of 80% real-world data and 20% synthetic mathematical processes [5]. That ratio should not be assumed for TimesFM 3.0; its official material states only that training used more than one trillion real and synthetic time points [6]. The breakdown below describes the 1.0 recipe:

1. Real-World Sources (80%): Google Trends, Wikipedia Pageviews, and open benchmark corpuses;
2. Synthetic Data (20%): Mixtures of \(\operatorname{ARMA}(p, q)\), Fourier sinusoids, changepoint polynomials, and random step functions. This teaches foundational geometric transformation independent of noisy domain bias.

3.4 Plain-English Glossary #

• Patch Token: High-dimensional continuous vector representing 32 consecutive observations;
• Context Window: Total historical lookback horizon ingested by self-attention in a single forward pass (scaling from 512 in 1.0 to 16,384 in 2.5);
• Future Covariates: Calendars and promotion schedules available across \(L+H\);
• Alternating Attention: Interleaving causal time attention with cross-variable attention (TimesFM 3.0);
• Causal Mask: Lower-triangular attention mask preventing future information leakage;
• Residual Block: Skip connections preventing gradient vanishing across 20 decoder layers;
• RevIN: Instance normalization stripping and restoring scale;
• RPM: Random prefix masking enabling seamless arbitrary-length inference;
• Pinball Loss: Asymmetric loss for learning conditional quantiles; empirical calibration still requires rolling evaluation.

4. Engineering Practice and Evaluation #

Having examined the underlying architecture and mathematical principles, algorithmic engineers focus on core operational questions: How do we deploy TimesFM in production? How do we handle promotional pricing and closures? How do we scale with minimal infrastructure overhead, and where do architectural limitations and licensing guardrails lie?

Note: The cloud implementation snippets in this chapter are based on the Google Cloud Official TimesFM Tutorial. Because this capability is currently governed by Pre-GA Offerings Terms, APIs and SQL syntax may evolve; readers should refer to the latest official documentation prior to production rollout.

4.1 Local Zero-Shot Inference #

For local workstations, the Apache-2.0 licensed TimesFM 2.5 (200M PyTorch) is an open-source candidate baseline. Its 200M FP32 parameters occupy roughly 800 MB, but total runtime memory is higher because of model, activation, and framework overhead. A light CPU experiment should plan for at least about 4 GB and measure usage for its context length and batch size:

import numpy as np
import timesfm

# 1. Load pre-trained TimesFM 2.5 (200M PyTorch, Apache-2.0 license)
model = timesfm.TimesFM_2p5_200M_torch.from_pretrained(
    "google/timesfm-2.5-200m-pytorch"
)

# 2. Compile forecast configuration
model.compile(
    timesfm.ForecastConfig(
        max_context=1024,
        max_horizon=28,
        normalize_inputs=True,
        use_continuous_quantile_head=True,
        return_backcast=True,  # Required by the later XReg call
    )
)

# 3. Supply variable-length sequences
inputs = [
    np.sin(np.linspace(0, 20, 365)) * 15 + 100,
    np.array([12.0, 15.0, 14.0, 18.0, 22.0, 25.0, 20.0,
              16.0, 19.0, 21.0, 24.0, 28.0, 30.0, 26.0]),
]

# 4. Zero-shot inference delivering point and quantile corridors
point_forecast, quantile_forecast = model.forecast(
    horizon=28,
    inputs=inputs,
)

4.2 XReg Covariate Engineering #

Pure univariate inference cannot anticipate scheduled marketing events. TimesFM provides XReg (In-Context Ridge Regression) to seamlessly inject business signals across history and forecast horizons (\(L + H\)). In production, engineering teams can select between two decoupled execution patterns:

1. Pattern 1 (xreg + timesfm, Default Recommended):
   • Execution Flow: Fits local ridge regression over the historical lookback to estimate promotional and closure impacts \(\hat{\boldsymbol{\beta}}\), extracting smooth residuals \(\mathbf{r} = \mathbf{y} - \mathbf{X}\hat{\boldsymbol{\beta}}\). TimesFM performs zero-shot forecasting on these stationary residuals, and future planned covariates \(\mathbf{X}_{\text{future}}\hat{\boldsymbol{\beta}}\) are superimposed over the forecast horizon.
   • Potential Advantage: Separates a linear covariate contribution from the residual time series, which may reduce interference from promotional pulses. The benefit should be validated against other fusion methods under the same information set.
2. Pattern 2 (timesfm + xreg, Alternative Mode):
   • Execution Flow: TimesFM first generates rolling baseline forecasts across the historical context, calculates unexplained residuals, and fits local ridge regression on the remaining delta.
   • Best For: Scenarios dominated by natural periodicity where weather and discounts only introduce subtle adjustments.

point_forecast, xreg_forecast = model.forecast_with_covariates(
    inputs=[history_sales],
    dynamic_numerical_covariates={"discount": [discount_series]},
    dynamic_categorical_covariates={
        "is_open": [is_open_series],
        "is_holiday": [holiday_series],
    },
    static_categorical_covariates={"category": ["grocery"]},
    xreg_mode="xreg + timesfm",
)

TimesFM 2.5’s XReg path also requires the optional timesfm[xreg] dependencies. The decomposition is interpretable, but its benefit depends on the linear assumption, feature quality, and evaluation protocol. TimesFM 3.0 instead integrates covariates inside the network through variate attention [6].

4.3 TimesFM 3.0 Python Adapter in the Companion Benchmark #

The companion retail benchmark does not call the archived BigQuery SQL experiment. It runs the TimesFM 3.0 Python checkpoint directly, pinning google/timesfm-3.0-pytorch to revision 43046b85ec22d584a13f8098c2ed39c889e129c2. The benchmark uses CPU inference with batch size 1. Pinning the revision does not claim that this checkpoint will remain the best release; it prevents a later rerun from silently downloading different weights.

At each forecast origin, the adapter performs four steps:

1. Build a regular daily context: a historical label is valid only when the store is open, no stockout censoring occurred, and y is present. The adapter masks invalid positions, then forward-fills and back-fills them on the regular daily grid. A history with no valid label fails instead of returning a synthetic forecast.
2. Construct the declared information set: the univariate configuration receives only the historical target. The known-covariate configuration also receives a matrix shaped as features \(\times\) (history + 28-day horizon). It includes opening status, calendar and Fourier terms, Victorian holidays, and scenario-specific business plans. Future y, realized stockout, conditional_mean, latent_demand, and other diagnostic truth columns are forbidden.
3. Call the native 3.0 interface: the process caches the model so that 13 folds do not reload the roughly 1.2 GB checkpoint. The core call is:

result = model.predict(
    context=context,
    horizon=len(future),
    past_future_covariates=covariates,
    return_quantiles=True,
    sort_quantiles=True,
)

4. Apply a common output contract: result.forecast is treated as the point forecast, while the first and ninth quantile columns become P10 and P90. Negative values are clipped to zero, and known closure days are set to zero. Repository tests perturb future sales, stockout flags, and diagnostic truth to verify that those changes cannot alter model inputs or predictions.

Part 2 therefore compares the Python TimesFM 3.0 zero-shot configurations with other frozen configurations under a unified evaluation protocol; it does not compare a BigQuery model with local models. Toolkits still use different internal training mechanisms. Comparability comes from common target dates, information boundaries, valid-label rules, and predeclared comparison roles—not from pretending that every model follows the same algorithm.

4.4 BigQuery Cloud Deployment #

Google embedded TimesFM directly into BigQuery ML as the AI.FORECAST table operator [7], eliminating GPU container orchestration:

SELECT
  store_id,
  sku_id,
  forecast_timestamp AS business_date,
  forecast_value AS yhat_point,
  prediction_interval_lower_bound AS yhat_p05,
  prediction_interval_upper_bound AS yhat_p95
FROM
  AI.FORECAST(
    (
      SELECT
        store_id,
        sku_id,
        business_date,
        clean_demand_units
      FROM
        `retail_prod_dw.daily_store_sku_demand`
      WHERE
        business_date <= DATE '2025-12-03'
    ),
    data_col => 'clean_demand_units',
    timestamp_col => 'business_date',
    id_cols => ['store_id', 'sku_id'],
    model => 'TimesFM 3.0',
    horizon => 28,
    confidence_level => 0.90
  )
ORDER BY
  store_id, sku_id, business_date;

4.5 Strengths and Architectural Highlights #

1. Version-Specific Long-Patch or CPM Decoding: TimesFM 1.0–2.5 use \(p=32 \to h=128\) to reduce rollout steps; TimesFM 3.0 combines a 64-step output patch with CPM. Both designs reduce repeated forward passes and opportunities for error accumulation, but neither guarantees millisecond latency or eliminates long-horizon error.
2. Pragmatic Covariate Decoupling via XReg: Decoupling promotional regression from Transformer feature spaces enables immediate feature injection without fine-tuning billions of parameters.
3. Lower Delivery Friction via BigQuery: A native SQL operator can reduce data movement and model-serving work. Cost and end-to-end latency still require measurements from query job statistics under the intended scale and concurrency.

4.6 Practical Limits and Licenses #

1. Complex Non-Linear Interactions Need a Separate Comparison: TimesFM 2.5 XReg is a local linear ridge model and cannot express every multi-way interaction, cross-item substitution, or hard monotonic constraint. It should be compared with controllable supervised models such as LightGBM under the same information set rather than ranked from architecture alone.
2. Intermittent and Zero-Inflated Demand Requires Separate Validation: Continuous patch models can regress toward small fractional forecasts on ultra-sparse series. That behavior is a reason to compare them with Croston, hurdle, and count-aware tree models across multiple origins, not proof that every patched model must fail.
3. Open-Source Licensing Boundaries: While TimesFM 1.0, 2.0, and 2.5 checkpoints are licensed under Apache-2.0, public TimesFM 3.0 PyTorch weights carry a non-commercial license (timesfm-non-commercial-license-v1.0) [6][9], restricting unmanaged self-hosted weights in commercial production (commercial compliance requires Google Cloud BigQuery AI.FORECAST or Vertex AI managed services). For engineering teams building private on-premise clusters, TimesFM 2.5 (200M) or alternative Apache-2.0 models like Chronos-2 [10] and Sundial [11] remain the primary compliant options.

5. Conclusion and Adoption Guide #

TimesFM evolved along two related but distinct paths. Version 1.0 established the \(32 \to 128\) asymmetric patch decoder; version 2.0 scaled to 500M parameters and a 2,048-step context; version 2.5 returned to a 200M backbone while extending context to 16,384 and adding an optional continuous quantile head. Version 3.0 then changed the decoding contract with 330M parameters, \(32 \to 64\) patches, CPM, and native multivariate attention [6]. Teams comparing foundation models should inspect patching, normalization, decoding, covariate contracts, and licensing separately rather than project one generation’s tensor path onto every release.

For engineering teams modernizing production forecasting systems, adopt three pragmatic guidelines:

1. Establish a Dual-Track Architecture (3.0 for Research/Managed Services vs. 2.5 for Apache-2.0 On-Premise):
   • Research and Cloud-Native Batch Serving (Candidate: TimesFM 3.0): When native multivariate inputs or future covariates matter, use a pinned Python revision for teaching or non-commercial evaluation, or assess the managed BigQuery service. Both paths require the organization’s own rolling backtests, interval metrics, cost measurements, and latency tests;
   • Private Clusters and Open-Source Compliance (Candidate: TimesFM 2.5): Self-hosted systems can evaluate the Apache-2.0 version 2.5 weights. Roughly 800 MB is the size of 200M FP32 parameters alone, not a complete runtime memory requirement.
2. Deploy TimesFM as a Rapid Zero-Shot Challenger: Before spending weeks crafting manual feature pipelines, establish an out-of-the-box baseline with TimesFM on clean historical data. If heavily tuned supervised models fail to beat zero-shot foundation model extrapolation, the feature engineering strategy or underlying data quality likely requires fundamental reassessment.
3. Treat Short-History Forecasting as a Separate Validation Regime: New products may not support long lag features, so compare TimesFM with seasonal-naive and business-informed statistical baselines. Random prefix masking explains how earlier TimesFM versions accept contexts that are not exact patch multiples; it does not guarantee superior cold-start accuracy. Use several launch cohorts and rolling origins to decide whether and when a model handoff is justified.

Next in Part 2: Time Series Foundation Models - Google TimesFM (Part 2): Synthetic Retail Experiments and Forecast Analysis runs the Python adapter over four five-year synthetic retail scenarios, using 13 non-overlapping 28-day forecast windows under expanding-history and fixed 104-week protocols. It separates the TimesFM univariate baseline, the known-covariate ablation, the broader model-family benchmark, and intermittent-demand event metrics while limiting conclusions to one fixed synthetic seed and the frozen configurations.

References #

[1] G. Woo et al. GIFT-Eval: A benchmark for general time series forecasting evaluation. arXiv:2406.06688, 2024. https://arxiv.org/abs/2406.06688

[2] O. Shchur et al. fev-bench: A unified framework and benchmark for time series forecasting evaluation. arXiv:2509.26468, 2025. https://arxiv.org/abs/2509.26468

[3] Z. Qiao et al. It’s TIME: Towards the next generation of time series forecasting benchmarks. In Proceedings of ICML, PMLR 306, 2026. https://arxiv.org/abs/2602.12147

[4] A. F. Ansari et al. Chronos: Learning the language of time series. Transactions on Machine Learning Research (TMLR), 2024. https://arxiv.org/abs/2403.07815

[5] A. Das, W. Kong, R. Sen, and Y. Zhou. A decoder-only foundation model for time-series forecasting. In Proceedings of ICML, PMLR 235, pp. 10148–10167, 2024. https://arxiv.org/abs/2310.10688

[6] A. Jain and R. Sen. TimesFM-3: A zero-shot foundation model for multivariate forecasting. Google Research Blog, 2026. https://research.google/blog/timesfm-3-a-zero-shot-foundation-model-for-multivariate-forecasting/

[7] Google Cloud. The AI.FORECAST function in BigQuery ML. Google Cloud Documentation, 2026. https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-ai-forecast

[8] T. Kim et al. Reversible instance normalization for accurate time-series forecasting against distribution shift. In ICLR, 2022. https://openreview.net/forum?id=cGDAkQo1C0p

[9] Google Research. TimesFM: Time Series Foundation Model Repository. GitHub, 2026. https://github.com/google-research/timesfm

[10] A. F. Ansari et al. Chronos-2: From univariate to universal forecasting. arXiv:2510.15821, 2025. https://arxiv.org/abs/2510.15821

[11] Y. Liu et al. Sundial: A family of highly capable time series foundation models. In Proceedings of ICML, PMLR 267, pp. 39295–39317, 2025. https://arxiv.org/abs/2502.00816