Kelly Criterion from Calibrated Forecasts: How to Size Positions When p_up Estimates Are Noisy?

A trader who has a model producing p_up, a calibrated probability of a favorable move for a given symbol and horizon, faces an immediate follow-up question: how much capital to risk on each signal. Kelly and high-fractional Kelly strategies have very risky short-term performance [4]. This matters because a retail trader typically has a finite horizon, limited capital, and cannot tolerate the kind of drawdown that a theoretically growth-optimal bettor might accept over an infinite horizon. Simulations found a consistent trade-off between growth and security as a function of the bet size selected by different strategies [4]. Regardless of how favorable the investment opportunities are or how long the finite horizon is, a sequence of bad scenarios can produce very poor final wealth outcomes and a loss of most of the investor's initial capital [4].

The problem is compounded because p_up is an estimate, not the true probability. Empirically derived Kelly bets based on sampled data can differ dramatically from Kelly bets based on the true return distribution [5]. In the example studied, the data-based practitioner judged the bet highly favorable and selected a large optimal betting fraction, whereas the theoretician using the true distribution selected little or no betting [5]. This is a direct warning for any pipeline that estimates p_up from a classifier trained on historical bars: the estimate can look good enough to justify a large bet while the true edge, if it were known, would justify none.

Estimation error is not unique to win-probability models. Estimation misspecifications in portfolio inputs can cause strong deviations from optimality and inferior out-of-sample performance [8]. Portfolio weights are extremely sensitive to changes in expected returns, which are more difficult to estimate than return covariances [8]. Even global minimum-variance portfolios, which do not use expected returns at all, can have significant estimation errors in their portfolio weights when covariance estimates are uncertain, especially in high-dimensional settings [8]. This tells a builder that noise in p_up is one instance of a broader estimation problem that plagues sizing decisions generally.

Finally, transaction costs and ruin are both active research topics on their own, separate from the classical no-cost Kelly result. In the presence of market frictions, conventional semimartingale and stochastic-integral analysis does not work in the general setting with transaction costs [7]. A lifetime-ruin investment problem with proportional transaction costs can be formulated as minimizing the probability that ruin occurs before death [10]. These are hard, separately studied problems; none of the cited work solves them jointly with noisy p_up estimation, so a builder should treat each adjustment as a separate, testable module rather than assume they compose cleanly.

In its simplest form, the Kelly criterion specifies the optimal fraction of capital to wager [5]. Under fair odds, independent and identically distributed (i.i.d.) horse-race market assumptions, and no transaction costs, Kelly proved the optimality of proportional betting strategies [3]. Under the same fair-odds assumptions, the asymptotic growth rate of cumulative wealth is maximal when the fraction of capital bet on each horse is proportional to its true winning probability [3]. The Kelly criterion calls for maximization of the expected logarithmic growth of wealth [13]. The classical Kelly betting problem can be solved using standard concave programming [13].

These are strong but conditional results. The proof depends on fair odds, i.i.d. outcomes, and no transaction costs [3]. A retail trader's environment, with bid-ask spreads, non-i.i.d. returns across regimes, and a classifier's p_up rather than a known true probability, violates every one of these conditions to some degree. This does not mean Kelly's proportional-betting idea is useless; it means the formula must be adapted, and each adaptation needs its own justification rather than an appeal to the original proof.

As the optimal Kelly fraction increases, various risk measures can become unacceptably large [5]. This is the practical reason fractional Kelly exists: fractional strategies reduce the optimal Kelly fraction so that less capital is at risk on each bet [5]. The trade-off is explicit and simulated: simulations found a consistent trade-off between growth and security as a function of the bet size selected by different strategies [4]. A builder choosing a fraction is choosing a point on this growth-versus-security curve, not eliminating the trade-off.

A further complication is that Kelly theory itself can recommend abstaining. When the return random variable has unbounded support, Kelly theory can lead to no betting at all [5]. This matters for a p_up-driven system because if the underlying return distribution implied by the classifier's outputs has heavy or unbounded tails, the theoretically correct response, once true parameters are known, may be to size at zero, not to shrink toward a small positive fraction. A builder should not assume a positive Kelly fraction is always the right answer even before considering estimation error.

Several distinct mechanisms address the gap between the true edge and its noisy estimate. A Taylor-based approximation recasts the Kelly betting problem into quadratic programming and produces a closed-form approximate solution [13]. The Taylor-based approximate Kelly approach provides results concerning betting performance, expected cumulative gain or loss, its variance, expected growth, variance of logarithmic growth, and survivability [13]. This gives a builder a computable, closed-form object rather than a full concave-program solve for each new p_up estimate, at the cost of accepting the approximation's error relative to the exact classical solution.

A separate approach addresses estimation error directly at the portfolio-construction level. An ensemble-learning growth optimal portfolio strategy combines bootstrap aggregating and the random subspace method to mitigate estimation error [2]. Simulation analysis and out-of-sample comparisons with 10 competing strategies on four datasets found that the ensemble-learning growth optimal portfolio strategy was superior across extensive evaluation criteria [2]. This is evidence for portfolio-level growth-optimal strategies under estimation error, evaluated against 10 named competing strategies on four datasets [2]; it is not evidence about single-asset Kelly sizing from a p_up classifier, so a builder cannot transplant the specific numeric superiority claim to a single-symbol sizing problem without re-testing it there.

For covariance-based portfolio problems specifically, cross-validation is documented as a remedy for a related but distinct estimation problem. When the asset-to-sample-size ratio is high, the sample covariance and its inverse have greater estimation error because their eigenvalues can be over- or underestimated [8]. When the concentration ratio q=n/T exceeds 1, the sample covariance matrix becomes singular and its inverse cannot be calculated [8]. Linear shrinkage estimators combine the sample covariance matrix with a structured target matrix to provide a bias-variance trade-off [8]. A multi-fold cross-validation technique can be used to select tuning parameters for covariance-estimation methods within each method [8]. In an empirical analysis using four S&P 500 datasets, data-driven tuning-parameter selection through cross-validation improved the out-of-sample performance of the global minimum-variance portfolio [8]. These four findings are about covariance estimation for multi-asset minimum-variance portfolios, not about p_up estimation error for a single-instrument Kelly bet; a builder working with a single p_up signal per symbol per horizon should treat this as a template for cross-validating any tunable parameter (including a shrinkage factor applied to p_up itself), not as a directly transferable number.

Position sizing, independent of how the edge estimate is produced, has its own measured effect on outcomes. Position sizing substantially affects the risk and return characteristics of technical trading strategies [9]. A study comparing position-sizing methods found that relative position sizing has a severe impact on trading results compared with erratic positions [9]. Smaller trading fractions delivered the highest risk-adjusted returns in most of the simulated scenarios [9]. This is direct, if separate, support for shrinking the sizing fraction below the naive Kelly-implied value, echoing the fractional-Kelly logic from a different empirical angle.

A calibrated p_up is not static; it is re-estimated as new bars arrive, so the question of how often to update the bet size is itself a modeled decision in the literature. In the dynamic game studied, the bettor selects a wealth fraction at time zero, keeps the resulting proceeds at risk for n steps, and updates the bet size only after those n steps [1]. This directly models a builder's practical choice: how many minute or daily bars to hold a sizing decision fixed before recomputing it from a fresh p_up.

The same study extends to costs and interest. The dynamic-game analysis explicitly examines the effects of accrued interest and transaction costs on optimal betting performance [1]. This is one of the few pieces of evidence that ties betting frequency directly to transaction costs within a single framework, which matters for anyone deciding whether to re-size after every minute bar (high frequency, more cost drag) or every daily bar (lower frequency, less cost drag).

The study does not fully resolve the general case. For general return distributions, the study states conjectures about how optimal performance changes with the update interval and about when a low-frequency bettor can match the performance of a high-frequency bettor [1]. Because these are stated as conjectures, a builder should treat any specific claim about

Transaction costs change the mathematical structure of the investment problem, not just its numeric result. In the transaction-cost market model, the riskless bond is the constant numéraire and the risky stock can be bought at the ask price S and sold at the bid price (1minus lambda)S [7]. This bid-ask spread formulation, with lambda as the proportional cost parameter, is the standard way the cited literature represents costs, and it maps directly onto a builder's need to subtract a spread-based cost from each simulated trade.

In the presence of market frictions, conventional semimartingale and stochastic-integral analysis does not work in the general setting with transaction costs [7]. With proportional transaction costs, acceptable portfolios can be defined through a consistent price system so that liquidation-value processes remain above stochastic thresholds [7]. These are results from continuous-time duality theory; they establish that transaction costs require a genuinely different analytical apparatus, not a simple subtraction from the no-cost Kelly formula. A builder should not assume that inserting a cost term into the classical Kelly log-growth objective and re-optimizing recovers the correct answer; the cited theory suggests the problem structure itself changes under costs.

Ruin is addressed as its own formal problem, again with transaction costs present. A lifetime-ruin investment problem with proportional transaction costs can be formulated as minimizing the probability that ruin occurs before death [10]. This gives a builder a named objective, minimize probability of ruin before a horizon, as an alternative to maximizing expected log growth; the two objectives are not the same, and this evidence base does not report a study that optimizes both jointly with a noisy p_up input.

Because these results come from different papers with different assumptions (no-cost fair-odds Kelly in [3], frequency and cost effects in [1], continuous-time duality with frictions in [7], and lifetime ruin in [10]), a builder cannot simply add a cost term to the classical Kelly formula and expect the guarantees of any one paper to carry over. Each of these frameworks should be treated as a distinct model to be tested separately against the trader's own cost structure and data, not as interchangeable derivations of the same number.

Expected log growth, the classical Kelly objective, is not the only thing a builder should measure. Maximizing expected growth can still produce significant drawdowns that make a strategy unsustainable [12]. This is a direct warning that a system tuned purely to maximize expected growth, even with a correctly estimated p_up, can still ruin a trader through path-dependent losses before the long-run growth advantage materializes.

Path-dependent risk measures such as drawdown risk can be used to assess the risk of significant portfolio retracements [12]. This gives a builder a concrete class of metrics, drawdown-based, to compute alongside terminal wealth or growth rate. Monte Carlo simulation can be used to analyze the medium-term behavior of cumulative return paths under different return distributions [12]. This is the recommended tool for turning a fitted or assumed return distribution (which could be derived from a sequence of p_up estimates and realized outcomes) into a distribution of drawdown paths rather than a single expected number.

Heavier-tailed outcome distributions require additional care because the growth-optimal strategy may not be optimal from a practical risk perspective [12]. This connects back to the earlier point that Kelly theory can recommend no betting at all when return support is unbounded [5]; both pieces of evidence point in the same direction, that tail behavior of the return distribution (however it is estimated from p_up and realized moves) should directly inform whether a positive bet size is even appropriate, separately from how large that size should be.

Taken together, these findings mean a builder's evaluation should not stop at reporting terminal wealth or an average growth rate. Reporting a maximum drawdown, or a distribution of drawdowns across simulated paths, is a distinct and necessary measurement, not an optional extra, because a strategy can maximize expected growth and still be practically unsustainable [12].

None of the cited work directly measures the joint effect of noisy p_up estimation, fractional Kelly shrinkage, drawdown floors, and cost-aware thresholds together on ruin probability. The classical optimality proof for Kelly betting holds under fair odds, i.i.d. outcomes, and no transaction costs [3], conditions that a retail p_up-driven system will not satisfy exactly. The warning that empirically derived Kelly bets can differ dramatically from true-distribution Kelly bets is stated with

1. Define the estimation unit. Treat each (symbol, horizon) pair as producing one p_up estimate at a given made_at timestamp, matching the forecasts table schema (symbol, horizon, made_at, q10, q50, q90, p_up). Store every p_up alongside the realized outcome (did price move favorably over that horizon) so you can later check calibration.

2. Compute the raw Kelly fraction. For a binary win/lose bet with win probability p_up and payoff odds b implied by q10/q50/q90, the classical Kelly fraction f* maximizes expected log growth [13], solvable by standard concave programming for the exact case [13], or by the Taylor-based quadratic approximation for a fast closed-form estimate [13].

3. Apply a fractional Kelly multiplier. Because the optimal Kelly fraction can produce unacceptably large risk measures [5], multiply f* by a fraction c in (0,1). Fractional strategies reduce the fraction so less capital is at risk per bet [5]; there is no cited number for the

The script below reads bars and forecasts from data.sqlite, computes a fractional-Kelly position size from p_up (implementing claims C, D, E, T, U), applies a transaction-cost haircut modeled on the ask/bid spread structure of claim Q, runs a Monte Carlo simulation of cumulative growth paths and drawdowns (implementing claims AJ, AK, AI), and compares a fractional-Kelly strategy against a full-Kelly baseline and a fixed-fraction baseline, printing terminal wealth, max drawdown, and ruin probability for each. Baseline to beat: full Kelly (fraction multiplier 1.0) should show higher ruin probability and larger max drawdown than fractional Kelly (multiplier 0.5), matching the growth-versus-security trade-off reported in claim G. Run it with: python kelly_note.py. If data.sqlite has no forecasts, the script synthesizes a small deterministic dataset so it still runs standalone.

import os
import sqlite3
import numpy as np
import pandas as pd

RNG = np.random.default_rng(42)

DB_PATH = os.environ.get("QOURAT_DB", "data.sqlite")


def load_forecasts(db_path):
    # Reads the forecasts table. Input: sqlite file path.
    # Output: DataFrame with columns symbol, horizon, made_at, q10, q50, q90, p_up.
    if not os.path.exists(db_path):
        return None
    con = sqlite3.connect(db_path)
    try:
        df = pd.read_sql_query("select * from forecasts", con)
    except Exception:
        df = None
    con.close()
    return df


def load_bars(db_path, tf="1d"):
    # Reads the bars table for a given timeframe.
    # Input: sqlite path, timeframe string '1m' or '1d'.
    # Output: DataFrame with columns symbol, tf, ts, open, high, low, close, volume.
    if not os.path.exists(db_path):
        return None
    con = sqlite3.connect(db_path)
    try:
        df = pd.read_sql_query(
            "select * from bars where tf = ?", con, params=(tf,)
        )
    except Exception:
        df = None
    con.close()
    return df


def synth_forecasts(n=250):
    # Builds a small deterministic synthetic dataset if data.sqlite is empty
    # or missing, so the script always runs end to end.
    p_up = np.clip(RNG.normal(0.55, 0.08, size=n), 0.05, 0.95)
    q50 = np.zeros(n)
    q10 = q50 - 0.02
    q90 = q50 + 0.02
    made_at = pd.date_range("2023-01-01", periods=n, freq="D")
    return pd.DataFrame(
        {
            "symbol": ["AAPL"] * n,
            "horizon": [1] * n,
            "made_at": made_at,
            "q10": q10,
            "q50": q50,
            "q90": q90,
            "p_up": p_up,
        }
    )


def kelly_fraction(p_up, win_return, lose_return):
    # Implements claim T and claim U: classical Kelly fraction for a binary
    # bet with win probability p_up, win multiple win_return (fractional
    # gain if the bet wins) and loss multiple lose_return (fractional loss
    # if the bet loses, positive number). Standard concave-programming
    # solution for this simple binary case: f* = p/lose_return -
    # (1-p)/win_return, clipped to [0, 1].
    f = p_up / lose_return - (1.0 - p_up) / win_return
    return np.clip(f, 0.0, 1.0)


def apply_cost_haircut(f, spread_frac):
    # Implements claim Q: the risky asset is bought at the ask and sold at
    # the bid, so a round trip costs approximately spread_frac of the
    # traded notional. We shrink the Kelly fraction by the cost as a
    # simple, transparent adjustment: less capital should be committed
    # when the round-trip cost eats into the edge.
    return np.clip(f * (1.0 - spread_frac), 0.0, 1.0)


def simulate_paths(f_series, win_return, lose_return, outcomes, n_paths=2000, start_wealth=1.0):
    # Implements claim AK (Monte Carlo simulation of cumulative return
    # paths) and supports computing claim AJ (drawdown) and claim AI
    # (growth vs sustainability). Inputs: per-step fraction f_series
    # (array, one value per historical bet), win_return, lose_return
    # (scalars), outcomes (array of 0/1, 1 = win, used to resample bet
    # results with replacement to build many paths). Output: array of
    # shape (n_paths, n_steps+1) of wealth trajectories.
    n_steps = len(f_series)
    wealth = np.full((n_paths, n_steps + 1), start_wealth, dtype=float)
    for t in range(n_steps):
        f = f_series[t]
        # Resample outcomes with replacement to build many possible paths
        wins = RNG.choice(outcomes, size=n_paths)
        gains = np.where(wins == 1, f * win_return, -f * lose_return)
        wealth[:, t + 1] = wealth[:, t] * (1.0 + gains)
        wealth[:, t + 1] = np.maximum(wealth[:, t + 1], 0.0)
    return wealth


def compute_max_drawdown(wealth_paths):
    # Implements claim AJ: max drawdown, the worst peak-to-trough decline in
    # each simulated path. Input: array (n_paths, n_steps+1). Output: array
    # (n_paths,) of max drawdown fractions.
    n_paths = wealth_paths.shape[0]
    mdd = np.zeros(n_paths)
    for i in range(n_paths):
        path = wealth_paths[i, :]
        running_max = np.maximum.accumulate(path)
        drawdown = (running_max - path) / np.maximum(running_max, 1e-9)
        mdd[i] = drawdown.max()
    return mdd


def compute_ruin_prob(wealth_paths, threshold=0.01):
    # Ruin probability: fraction of paths that drop below threshold of
    # starting wealth. Input: array (n_paths, n_steps+1). Output: scalar
    # probability.
    terminal_wealth = wealth_paths[:, -1]
    return (terminal_wealth  0.5, treat as a win. This is used
    # only to set an empirical resampling probability for the simulation,
    # not to leak future information.
    outcomes = (p_up > 0.5).astype(int)

    full_kelly_f = kelly_fraction(p_up, win_return, lose_return)

    # claim E: fractional Kelly multiplier
    frac_multiplier = 0.5
    fractional_kelly_f = full_kelly_f * frac_multiplier

    # claim Q: cost haircut using an assumed round trip spread of 0.5 percent
    spread_frac = 0.005
    cost_aware_f = apply_cost_haircut(fractional_kelly_f, spread_frac)

    # fixed-fraction baseline, independent of p_up
    fixed_f = np.full_like(full_kelly_f, 0.05)

    print(f"n bets: {len(p_up)}, win_return={win_return:.4f}, lose_return={lose_return:.4f}")
    print()

    results = {}
    results["full_kelly"] = evaluate_strategy(
        "Full Kelly (baseline, claim C/T/U)", full_kelly_f, win_return, lose_return, outcomes
    )
    print()
    results["fractional_kelly"] = evaluate_strategy(
        "Fractional Kelly x0.5 (claim E)", fractional_kelly_f, win_return, lose_return, outcomes
    )
    print()
    results["cost_aware"] = evaluate_strategy(
        "Fractional Kelly + cost haircut (claim Q)", cost_aware_f, win_return, lose_return, outcomes
    )
    print()
    results["fixed_fraction"] = evaluate_strategy(
        "Fixed 5 percent baseline", fixed_f, win_return, lose_return, outcomes
    )

    print()
    print("Check: fractional Kelly should show lower ruin probability and lower")
    print("mean max drawdown than full Kelly (claims F, G). If it does not,")
    print("the frac_multiplier or cost assumptions should be revisited.")


if __name__ == "__main__":
    main()