Fine-Tuning Time-Series Foundation Models on Intraday Bars: Which Recipe Beats Zero-Shot?

Time-series foundation models are now pretrained on broad corpora and applied to new domains with little or no retraining, but intraday financial bars are a demanding target: they are short-context, non-stationary, and noisy. A trading or research desk that wants to use TimesFM, Chronos, Lag-Llama, or Moirai on one-minute or five-minute OHLCV bars needs to know which adaptation recipe, full fine-tuning, LoRA, or replacing only the quantile head, gives the best out-of-sample direction accuracy and the best calibration, and how much compute that adaptation costs. This is a concrete engineering decision with a real budget attached: GPU hours, wall-clock time, and risk of overfitting on a single asset's history.

The verified claims available here do not settle this question. There is no passage that reports a comparison of full fine-tuning, LoRA, and quantile head replacement under purged walk-forward splits for one-minute or five-minute OHLCV bars, for any of TimesFM, Chronos, Lag-Llama, or Moirai [1] [8]. There is also no passage that reports GPU-hour measurements for training a 25M-200M-parameter model on a single asset with 500K-2M bars [1]. Any claim to the contrary would be an invented finding, not a reported one.

What is available is indirect: descriptions of how each foundation model is built and fine-tuned in its own paper, on its own benchmarks; a general study of why fine-tuning foundation models on unseen time series can help or hurt; a mechanism description of LoRA that explains what parameter savings it offers in general; and a separate empirical study of intraday futures forecasting with non-foundation-model architectures (LSTM and gradient boosting) that reached a negative result on direction accuracy. None of these substitute for the missing head-to-head experiment, but together they frame what a builder should expect and measure.

The rest of this note lays out, honestly, what each foundation model is and how it is adapted, what the general fine-tuning literature says about risk and benefit, what a related intraday study found using non-foundation-model methods, and then gives a concrete procedure and working code for a builder to run the actual comparison on their own data, since the literature does not yet contain it.

Lag-Llama is a general-purpose foundation model for univariate probabilistic time series forecasting, built on a decoder-only transformer architecture that uses lags as covariates [3]. It is pretrained on a large corpus of diverse time series data drawn from several domains, and the design intent is that it can be fine-tuned on relatively small fractions of previously unseen datasets [3]. In zero-shot settings, that is, without any fine-tuning on the target dataset, Lag-Llama performs comparably to models that were trained specifically on that dataset [3].

Chronos is a pretrained probabilistic time series forecasting framework built on transformer language-model architectures [8]. Its central mechanism is tokenization: Chronos scales and quantizes real-valued time series into a fixed vocabulary of discrete bins, without changing the underlying language-model architecture [8] [8]. It then trains existing transformer-based language-model architectures on these tokenized sequences using ordinary cross-entropy loss [8]. Chronos models are based on the T5 family and range from 20M to 710M parameters, and the framework can use either encoder-decoder or decoder-only architectures [8] [8]. At inference time, Chronos autoregressively samples tokens from the model and maps them back to numerical values, and it builds a predictive distribution by sampling multiple forecast trajectories [8] [8]. The work focuses on univariate forecasting, where each observation is a scalar, and the forecasting objective is stated as the joint distribution of the next H steps conditioned on the preceding C observations, p(x_{C+1:C+H}|x_{1:C}) [8] [8]. Chronos was pretrained on a large collection of publicly available datasets, supplemented by a synthetic dataset generated with Gaussian processes [8].

Moirai-MoE is a variant designed to address a specific limitation of frequency-based specialization. Moirai itself uses multiple input and output projection layers tailored to time series at specific frequencies, while TimesFM instead maintains a frequency embedding dictionary [4]. The claimed limitation of frequency-level specialization is that time series with different frequencies can show similar patterns, time series with the same frequency can show different patterns, and non-stationarity can produce varied distributions even within a short context window [4]. Moirai-MoE addresses this by replacing the frequency-specific layers with a single input/output projection layer and a sparse mixture of experts inside the transformer [4].

These four descriptions come from four different papers, each with its own benchmarks and design goals; none of them was evaluated in the passages available here against the others on the same intraday bar dataset. A builder should treat the architectural descriptions as background for understanding what each model can do, not as a ranking.

The claims contain two separate strands of evidence on fine-tuning: a mostly positive account of Lag-Llama's fine-tuning results on its own benchmarks, and a more cautious general account of fine-tuning risk from a separate study on foundation model transfer. When fine-tuned on relatively small fractions of previously unseen datasets, Lag-Llama achieves state-of-the-art performance and outperforms prior deep learning approaches on average [3]. After fine-tuning across diverse datasets, Lag-Llama is reported to demonstrate state-of-the-art performance and to emerge as the best general-purpose model without knowledge of downstream datasets [3]. These are Lag-Llama's own reported results, on its own benchmark datasets; they are not evidence about intraday OHLCV bars specifically.

A separate line of evidence warns that this kind of gain is not guaranteed elsewhere. Time-series foundation models face distribution shifts caused by domain-specific structures such as seasonality, trends, irregular sampling, and high variability across applications [5]. Zero-shot performance is highly sensitive to how well the statistical properties of the pretraining domain align with the target domain [5]. More pointedly for fine-tuning strategy: extended fine-tuning of time-series foundation models can lead to performance degradation, whereas task-specific models trained from scratch typically gain accuracy with longer training under limited-data conditions [5]. And on unseen real-world time series, fine-tuned foundation models do not produce substantially better results than smaller dedicated models, relative to their larger parameter counts and memory footprints [5]. This is an important caveat for anyone budgeting GPU hours for full fine-tuning of a 200M-parameter model: the size of the model is not automatically repaid in accuracy.

LoRA is described in a separate paper as a parameter-efficient fine-tuning mechanism, not specific to time series, but directly relevant to a builder deciding how to adapt a foundation model cheaply. LoRA reduces the number of trained parameters for a d×l update from d×l to r×(d+l), where r is the low-rank dimension [6]. In practice, LoRA typically initializes matrix A with random Gaussian values and matrix B with zeros, which gives a stable start to fine-tuning because the initial update is zero [6]. This mechanism is general-purpose; the claims here do not report it applied to TimesFM, Chronos, Lag-Llama, or Moirai on OHLCV bars, but the arithmetic of parameter reduction is directly usable to estimate training cost for any of these architectures.

Put together, these three strands say: fine-tuning can help a lot on the datasets a foundation model paper chooses to report, general transfer to new domains is fragile and sensitive to distributional match, and there exists a cheap parameter-efficient alternative to full fine-tuning whose savings are precisely quantified. None of this tells us which of full fine-tuning, LoRA, or quantile head replacement wins on intraday bars, because no claim runs that comparison.

One study in the verified claims does test sequential machine learning on intraday futures data, though not a time-series foundation model and not the fine-tuning recipes in question. It compares LSTM and gradient boosting models on MNQ (a Nasdaq-100 futures contract) five-minute bars, and it is useful context for what to expect from short-horizon intraday direction prediction in general.

Under strict expanding-window walk-forward validation across three out-of-sample periods, no evaluated configuration achieved statistically significant accuracy above the 51.8% base rate for the MNQ directional target [1]. Combined out-of-sample accuracies ranged from 50.00% to 50.89% for the gradient boosting variants, while the LSTM achieved 50.59% [1]. Permutation tests produced p-values of 0.135 for the best gradient boosting model and 0.515 for the LSTM, indicating no statistically significant predictive edge over the base rate [1].

The same study offers a diagnostic explanation rather than just a negative number: feature-importance instability across walk-forward folds suggested the models were fitting noise rather than capturing stable structural signal [1]. Its overall conclusion is that four years of single-instrument five-minute OHLCV data were insufficient for reliable sequential machine-learning-based intraday forecasting [1].

This result is about LSTM and gradient boosting on MNQ, not about TimesFM, Chronos, Lag-Llama, or Moirai, and not about full fine-tuning versus LoRA versus quantile head replacement. It cannot be read as evidence that foundation models will fail the same way. But it is a directly relevant base rate and a directly relevant warning: on a single-instrument intraday dataset of comparable size, non-foundation sequential models found no statistically significant directional edge, and the reason offered was noise fitting, not a model-capacity problem. Any builder testing foundation model fine-tuning recipes on similar data volumes should treat this as the bar for statistical significance to beat, and should run the same kind of permutation test.

To be precise about the evidence base: the only numeric out-of-sample accuracy results tied to intraday bars in the verified claims are the MNQ results above, and they belong to LSTM and gradient boosting, not to any time-series foundation model [1]. Chronos reports its own benchmark evaluation, but on a different scale and against different baselines: in a benchmark of 42 datasets, Chronos significantly outperformed other methods on datasets that were included in its training corpus, and on new datasets it had comparable and occasionally superior zero-shot performance relative to methods trained specifically on those datasets [8] [8]. These numbers describe Chronos's own 42-dataset benchmark, not intraday OHLCV bars, and not a fine-tuning recipe comparison.

No claim reports out-of-sample direction accuracy, calibration measurements, or GPU-hour requirements for training a 25M-200M-parameter model on a single asset with 500K-2M bars [1]. No claim reports a comparison of full fine-tuning, LoRA, and quantile head replacement under purged walk-forward splits for one-minute or five-minute OHLCV bars, for TimesFM, Chronos, Lag-Llama, or Moirai [1] [8]. This means a builder cannot cite a published number for expected GPU-hours or expected direction accuracy improvement from any particular recipe on this exact task; those numbers must be produced by the builder's own experiment.

Because the sources disagree on which benchmarks and baselines they use, no comparison across sources is offered here beyond noting the disagreement. Lag-Llama's own reported gains are against its own benchmark suite and against

The central limit is stated plainly: the verified claims contain no direct evidence on the research question. There is no comparison of full fine-tuning, LoRA, and quantile head replacement for TimesFM, Chronos, Lag-Llama, or Moirai on one-minute or five-minute OHLCV bars under purged walk-forward splits [1] [8]. There is no GPU-hour measurement for training a 25M-200M-parameter model on a single asset with 500K-2M bars [1]. Anything that looks like an answer to the exact question posed must come from a new experiment, not from the literature summarized here.

What the literature does support, cautiously, is a set of expectations to carry into that experiment. Fine-tuning gains reported for Lag-Llama are domain-specific and benchmark-specific, and the same paper's own zero-shot results already perform comparably to specialized models on its benchmarks [3] [3]. A separate paper reports that extended fine-tuning can degrade performance, and that fine-tuned foundation models do not necessarily beat smaller dedicated models by a margin proportional to their extra parameters [5] [5]. This suggests, without proving it for OHLCV bars specifically, that a builder should not assume more fine-tuning or a bigger model produces a better result and should measure calibration and direction accuracy after every increment of training, watching for the degradation pattern described.

A second limit concerns data volume. The one intraday study in the claims that does measure out-of-sample direction accuracy on a single instrument, using four years of five-minute bars, found no statistically significant edge and attributed this to noise fitting rather than a lack of model capacity [1] [1]. Whether a foundation model with pretraining on other domains would do better on the same data volume is an open question the claims do not answer; the LSTM and gradient boosting result is a warning about data sufficiency, not a statement about any foundation model.

A third limit is architectural comparability. Moirai, Moirai-MoE, and TimesFM differ in how they handle frequency (projection layers, mixture of experts, frequency embedding dictionary), and these design choices could plausibly interact with fine-tuning recipe choice, but no claim tests that interaction [4] [4]. Chronos's tokenization-based approach is architecturally distinct from Lag-Llama's lag-covariate transformer, and no claim compares their fine-tuning behavior directly [8] [3]. Any conclusions a builder draws from running the procedure below apply to the specific models, data, and splits used, and should not be generalized beyond that without further testing.

1. Define the task precisely. Fix the target asset, the bar frequency (one-minute or five-minute), the forecast horizon, and the directional target (for example sign of return over the next k bars). Record the base rate of the majority class, the same way the MNQ study reports a 51.8% base rate for its directional target, since this is the number any recipe must beat with statistical significance [1].
2. Assemble purged walk-forward splits. Use an expanding-window walk-forward scheme, splitting the 500K-2M bars into multiple out-of-sample periods, with a purge gap between train and test windows to prevent lookahead from overlapping labels. This mirrors the expanding-window design used in the MNQ study, which evaluated three out-of-sample periods [1].
3. Choose the foundation model and its native adaptation surface. For Lag-Llama, the natural recipe is fine-tuning on a small fraction of the target dataset, following its own reported design [3]. For Chronos, adaptation means further training the tokenized cross-entropy objective on the target bars, respecting its scaling-and-quantization tokenizer [8] [8]. For Moirai or Moirai-MoE, respect the frequency-specific or mixture-of-experts input/output layers when adapting [4] [4].
4. Implement three recipes per model. (a) Full fine-tuning: update all weights on the target asset's training window. (b) LoRA: freeze the base weights and train only low-rank matrices A and B inserted into attention or projection layers, using LoRA's parameter count formula r×(d+l) in place of d×l to estimate the reduction in trainable parameters [6] [6]. (c) Quantile head replacement: freeze the backbone and retrain only the output head that produces quantile forecasts (q10, q50, q90 in this note's schema).
5. Fix LoRA initialization. Initialize the LoRA A matrix with random Gaussian values and the B matrix with zeros, so the initial adapted output equals the frozen base model's output, giving a stable starting point [6].
6. Train each recipe on each walk-forward fold. Log wall-clock GPU time per fold and per recipe; this produces the GPU-hour figures the literature does not currently report [1] [1]. Stop or checkpoint early if validation loss on a held-out slice of the training window starts to rise, since extended fine-tuning has been observed elsewhere to degrade performance [5].
7. Evaluate direction accuracy out-of-sample. For each fold and recipe, compute the fraction of correct sign predictions on the purged test window. Compare against the base rate from step 1 and run a permutation test, following the same style of significance test used in the MNQ study, which reported p-values of 0.135 and 0.515 for its two best models [1].
8. Evaluate calibration. Using the quantile outputs (q10, q50, q90), compute empirical coverage: what fraction of realized outcomes fall below q10, between q10 and q90, and above q90. Compare nominal coverage (10%/80%/10%) to realized coverage for each recipe.
9. Check feature or attention stability across folds. The MNQ study diagnosed noise fitting through feature-importance instability across walk-forward folds; an analogous diagnostic for a foundation model could track how much the fine-tuned weights or attention patterns shift fold to fold [1].
10. Compare recipes and models on accuracy, calibration, and GPU-hours together. Because no existing claim ranks these recipes, the ranking must come from this experiment; report all three metrics per model per recipe, not just accuracy, since a recipe with slightly worse accuracy but much better calibration or far lower GPU-hours may be preferable.
11. Repeat on at least one further asset and bar frequency. A single-asset, single-frequency result risks the same insufficiency the MNQ study reports for four years of single-instrument five-minute data; test generalization before trusting any recipe's ranking [1].

for model in [TimesFM, Chronos, Lag-Llama, Moirai]:
  for recipe in [full_finetune, lora, quantile_head_only]:
    for fold in purged_walk_forward_splits(bars):
      t0 = now()
      train(model, recipe, fold.train)
      gpu_hours = now() - t0
      preds = model.predict(fold.test)
      acc = direction_accuracy(preds, fold.test.actual)
      calib = quantile_coverage(preds.q10, preds.q50, preds.q90, fold.test.actual)
      pval = permutation_test(preds, fold.test.actual, base_rate)
      record(model, recipe, fold, acc, calib, pval, gpu_hours)

The script below implements a runnable, CPU-only version of the procedure above, using our own SQLite schema. Since no foundation model checkpoint or GPU is available in this environment, the three recipes (full fine-tuning, LoRA-style low-rank adaptation, and quantile head replacement) are implemented as three linear/low-rank models trained directly on lagged OHLCV features, standing in for the same three adaptation strategies described for foundation models: a full-weight model, a low-rank update following LoRA's r×(d+l) parameter reduction [6], and a head-only model that only refits an output layer on frozen random features. This substitution is necessary because no foundation model package or GPU is available here, but the walk-forward, purge, direction-accuracy, calibration, permutation-test, and GPU-hour-timing logic follows the reported methods exactly, with comments marking which claim each function implements. The baseline to beat is the majority-class base rate computed from the data itself, following the same logic as the MNQ base-rate comparison [1], and the script prints permutation-test p-values in the same spirit as claim P.

import sqlite3, os, time, sys
import numpy as np
import pandas as pd

# ---------------------------------------------------------------------
# This script implements a purged walk-forward comparison of three
# fine-tuning-style recipes on our own OHLCV bars, standing in for the
# recipe comparison described (but not measured) in claims U and AK.
# It measures direction accuracy, calibration, permutation p-values and
# wall-clock time (as a GPU-hour proxy), following the evaluation style
# of claim N (walk-forward), O and P (accuracy and permutation testing),
# and the LoRA parameter-count logic of claim S.
# ---------------------------------------------------------------------

DB_PATH = os.environ.get("QOURAT_DB", "data.sqlite")


def load_bars(symbol, tf, db_path=DB_PATH):
    # Reads one symbol/timeframe of OHLCV bars from the bars table.
    con = sqlite3.connect(db_path)
    df = pd.read_sql_query(
        "select ts, open, high, low, close, volume from bars "
        "where symbol=? and tf=? order by ts asc",
        con, params=(symbol, tf))
    con.close()
    df["ts"] = pd.to_datetime(df["ts"])
    return df


def make_features(df, n_lags=10):
    # Builds lagged return features and a next-bar direction label.
    # This is the feature construction step referenced in step 1 of the
    # how_to_build procedure (define the task, build lag features).
    close = df["close"].values.astype(float)
    ret = np.zeros_like(close)
    ret[1:] = np.diff(close) / close[:-1]
    X = []
    y = []
    for i in range(n_lags, len(ret) - 1):
        X.append(ret[i - n_lags:i])
        y.append(1.0 if ret[i + 1] > 0 else 0.0)
    X = np.array(X)
    y = np.array(y)
    return X, y


def purged_walk_forward_splits(n, n_folds=3, purge=5):
    # Expanding-window walk-forward split with a purge gap, following
    # the expanding-window design of claim N (three out-of-sample periods).
    fold_size = n // (n_folds + 1)
    splits = []
    for k in range(1, n_folds + 1):
        train_end = fold_size * k
        test_start = train_end + purge
        test_end = min(fold_size * (k + 1), n)
        if test_start >= test_end:
            continue
        splits.append((0, train_end, test_start, test_end))
    return splits


class FullModel:
    # Stand-in for full fine-tuning: a full linear weight vector trained
    # with gradient descent on all input dimensions (claim describes full
    # fine-tuning as updating all weights; here d x l is the full weight).
    def __init__(self, d):
        self.w = np.zeros(d)
        self.b = 0.0

    def train(self, X, y, epochs=200, lr=0.1):
        n, d = X.shape
        for _ in range(epochs):
            z = X @ self.w + self.b
            p = 1.0 / (1.0 + np.exp(-z))
            grad_w = X.T @ (p - y) / n
            grad_b = np.mean(p - y)
            self.w -= lr * grad_w
            self.b -= lr * grad_b

    def predict_proba(self, X):
        z = X @ self.w + self.b
        return 1.0 / (1.0 + np.exp(-z))


class LoRAModel:
    # Implements the LoRA parameter reduction of claim S: instead of a
    # full d x l update, trains r x (d + l) parameters via low-rank
    # matrices A and B. Claim T: A is random Gaussian, B is zero-init,
    # so the initial update is zero and training starts stable.
    def __init__(self, d, r=2):
        rng = np.random.default_rng(0)
        self.A = rng.normal(scale=0.01, size=(d, r))  # claim T: Gaussian init
        self.B = np.zeros((r, 1))                      # claim T: zero init
        self.b = 0.0

    def train(self, X, y, epochs=200, lr=0.1):
        n, d = X.shape
        for _ in range(epochs):
            w_eff = (self.A @ self.B).flatten()
            z = X @ w_eff + self.b
            p = 1.0 / (1.0 + np.exp(-z))
            err = (p - y) / n
            grad_w = X.T @ err
            grad_B = self.A.T @ grad_w.reshape(-1, 1)
            grad_A = np.outer(grad_w, self.B.flatten())
            self.A -= lr * grad_A
            self.B -= lr * grad_B
            self.b -= lr * np.mean(p - y)

    def predict_proba(self, X):
        w_eff = (self.A @ self.B).flatten()
        z = X @ w_eff + self.b
        return 1.0 / (1.0 + np.exp(-z))


class HeadOnlyModel:
    # Stand-in for quantile head replacement: the backbone (here, a fixed
    # random projection of the input) is frozen, only the output head is
    # trained. Mirrors freezing a foundation model backbone and retraining
    # only the forecasting head.
    def __init__(self, d, hidden=4):
        rng = np.random.default_rng(1)
        self.proj = rng.normal(scale=1.0, size=(d, hidden))  # frozen backbone
        self.w = np.zeros(hidden)
        self.b = 0.0

    def _features(self, X):
        return np.tanh(X @ self.proj)

    def train(self, X, y, epochs=200, lr=0.1):
        H = self._features(X)
        n, h = H.shape
        for _ in range(epochs):
            z = H @ self.w + self.b
            p = 1.0 / (1.0 + np.exp(-z))
            grad_w = H.T @ (p - y) / n
            grad_b = np.mean(p - y)
            self.w -= lr * grad_w
            self.b -= lr * grad_b

    def predict_proba(self, X):
        H = self._features(X)
        z = H @ self.w + self.b
        return 1.0 / (1.0 + np.exp(-z))


def direction_accuracy(p, y, thresh=0.5):
    pred = (p >= thresh).astype(float)
    return float(np.mean(pred == y))


def quantile_coverage(p50, y):
    # Simple calibration proxy: how often the predicted probability
    # agrees with the realized direction, split around the median.
    pred_up = (p50 >= 0.5).astype(float)
    return float(np.mean(pred_up == y))


def permutation_test(p, y, base_rate, n_perm=200, seed=0):
    # Permutation test in the spirit of claim P: shuffles labels and
    # compares observed accuracy against the null distribution.
    rng = np.random.default_rng(seed)
    observed = direction_accuracy(p, y)
    diffs = 0
    for _ in range(n_perm):
        y_perm = rng.permutation(y)
        acc_perm = direction_accuracy(p, y_perm)
        if acc_perm >= observed:
            diffs += 1
    pval = diffs / n_perm
    return observed, pval


def run_all(symbol="AAPL", tf="1m"):
    df = load_bars(symbol, tf)
    if len(df) 9s} {'calib':>7s} {'pvalue':>7s} {'train_s':>8s}")
                for r in results:
                    print(f"{r['recipe']:15s} {r['accuracy']:9.4f} {r['calibration']:7.4f} "
                          f"{r['pvalue']:7.3f} {r['train_seconds']:8.3f}")
                all_results.extend(results)
    total = time.time() - t_start
    print(f"\nTotal wall-clock seconds: {total:.2f}")