التنبؤ التوافقي (Conformal Prediction) لتنبؤات السلاسل الزمنية: كيف نُغلّف كميات (quantiles) نموذج الأساس؟

الادعاءات والمراجعة

الادعاءات والمراجعة

1. methodمدعوم

   Adaptive conformal inference (ACI) wraps black-box methods that produce point predictions or estimated quantiles for the response.

   [2] Adaptive Conformal Inference Under Distribution Shift, section 1 Introduction

   “While exchangeability is a common assumption, there are many real-world applications in which we do not expect the marginal distribution of (Xt,Yt)(X_{t},Y_{t}) to be stationary. For example, in finance and economics market behaviour can shift drastically in response to new legis…”

2. methodمدعوم

   For a model producing lower and upper quantiles, conformal quantile regression uses the score S(X_t,y)=max{q^(X_t;α/2)−y,y−q^(X_t;1−α/2)}.

   [2] Adaptive Conformal Inference Under Distribution Shift, section 1.1 Conformal inference

   “Suppose we are given a fitted regression model for predicting the value of YY from XX. Let yy be a candidate value for YtY_{t}. To determine if yy is a reasonable estimate of YtY_{t}, we define a conformity score S⁡(X,Y)S(X,Y) that measures how well the value yy conforms with the…”

3. methodمدعوم

   Sequential conformal prediction constructs C_t={y∈𝒴:s_t(x_t,y)≤q_t}, where q_t estimates the 1−α quantile of the score at time t.

   [1] Conformal PID Control for Time Series Prediction, section 1 Introduction

   “The challenge in the sequential setting is as follows. We seek to invert the score function to construct a conformal prediction set, 𝒞t={y∈𝒴:st​(xt,y)≤qt},\mathcal{C}_{t}=\{y\in\mathcal{Y}:s_{t}(x_{t},y)\leq q_{t}\}, (1) where qtq_{t} is an estimated 1−α1-\alpha quantile for the …”

4. factمدعوم

   In the online distribution-shift setting, the covariates and responses may form a potentially adversarial deterministic time series, and no probabilistic model for the data is assumed.

   [1] Conformal PID Control for Time Series Prediction, section 1 Introduction

   “Machine learning models run in production systems regularly encounter data distributions that change over time. This can be due to factors such as seasonality and time-of-day, continual updating and re-training of upstream machine learning models, changing user behaviors, and so …”

5. methodمدعوم

   A valid online conformal score can be any negatively oriented function s_t:𝒳×𝒴→ℝ, where lower values indicate greater forecast accuracy.

   [1] Conformal PID Control for Time Series Prediction, section 1 Introduction

   “Machine learning models run in production systems regularly encounter data distributions that change over time. This can be due to factors such as seasonality and time-of-day, continual updating and re-training of upstream machine learning models, changing user behaviors, and so …”

6. methodمدعوم

   The conformal PID update sets q_{t+1}=q^_{t+1}+r_t(∑_{i=1}^t(err_i−α)), where q^_{t+1} is any function of past x_i,y_i,q_i for i≤t.

   [1] Conformal PID Control for Time Series Prediction, section 1.1 Peek at results: methods

   “example is the tangent integrator rt​(x)=KI​tan⁡(x​log⁡(t)/(t​Csat))r_{t}(x)=K_{\text{I}}\tan(x\log(t)/(tC_{\text{sat}})), where we set tan⁡(x)=sign⁡(x)⋅∞\tan(x)=\mathrm{sign}(x)\cdot\infty for x∉[−π/2,π/2]x\notin[-\pi/2,\pi/2], and Csat,KI>0C_{\text{sat}},K_{\text{I}}>0 are cons…”

7. methodمرفوض

   Taking q^_{t+1}=ηg_t+g′_t recovers the formulation in which the scorecaster predicts the next score quantile.

   [1] Conformal PID Control for Time Series Prediction, section 1.1 Peek at results: methods

   “example is the tangent integrator rt​(x)=KI​tan⁡(x​log⁡(t)/(t​Csat))r_{t}(x)=K_{\text{I}}\tan(x\log(t)/(tC_{\text{sat}})), where we set tan⁡(x)=sign⁡(x)⋅∞\tan(x)=\mathrm{sign}(x)\cdot\infty for x∉[−π/2,π/2]x\notin[-\pi/2,\pi/2], and Csat,KI>0C_{\text{sat}},K_{\text{I}}>0 are cons…”

8. methodمدعوم

   Quantile tracking applies running online gradient descent to the quantile loss summed over past scores and is described as proportional control.

   [1] Conformal PID Control for Time Series Prediction, section 1.1 Peek at results: methods

   “Three design principles underlie our methods: 1. Quantile tracking (P control). Running online gradient descent on the quantile loss (summed over all past scores) gives rise to a method that we call quantile tracking, which achieves long-run coverage (2) under no assumptions exce…”

9. resultمدعوم

   Quantile tracking achieves long-run coverage under bounded scores without additional assumptions, even when the bound is unknown.

   [1] Conformal PID Control for Time Series Prediction, section 1.1 Peek at results: methods

   “Three design principles underlie our methods: 1. Quantile tracking (P control). Running online gradient descent on the quantile loss (summed over all past scores) gives rise to a method that we call quantile tracking, which achieves long-run coverage (2) under no assumptions exce…”

10. limitationمدعوم

    Unlike adaptive conformal inference (ACI), quantile tracking does not return infinite sets after a sequence of miscoverage events.

    [1] Conformal PID Control for Time Series Prediction, section 1.1 Peek at results: methods

    “Three design principles underlie our methods: 1. Quantile tracking (P control). Running online gradient descent on the quantile loss (summed over all past scores) gives rise to a method that we call quantile tracking, which achieves long-run coverage (2) under no assumptions exce…”

11. methodمدعوم

    Error integration incorporates the running sum ∑_{i=1}^t(err_i−α) into online quantile updates and is described as integral control.

    [1] Conformal PID Control for Time Series Prediction, section 1.1 Peek at results: methods

    “Three design principles underlie our methods: 1. Quantile tracking (P control). Running online gradient descent on the quantile loss (summed over all past scores) gives rise to a method that we call quantile tracking, which achieves long-run coverage (2) under no assumptions exce…”

12. resultمدعوم

    The error integration scheme achieves long-run coverage without any assumptions on the scores, including when the scores are unbounded.

    [1] Conformal PID Control for Time Series Prediction, section 1.1 Peek at results: methods

    “Three design principles underlie our methods: 1. Quantile tracking (P control). Running online gradient descent on the quantile loss (summed over all past scores) gives rise to a method that we call quantile tracking, which achieves long-run coverage (2) under no assumptions exce…”

13. methodمدعوم مع تحفظات

    Scorecasting trains a second model to predict the quantile of the next score, allowing the procedure to account for systematic score trends caused by seasonality, trends, or distributional changes.

    [1] Conformal PID Control for Time Series Prediction, section 1.1 Peek at results: methodsPassage says scorecasting trains 'a second model, namely, a scorecaster, to predict the quantile of the next score' and that it can 'account for systematic trends in the scores,' but the listed causes (seasonality, trends, distributional changes) are the claim's paraphrase—passage only says 'aspects of the data distribution, fixed or changing.'

    “Three design principles underlie our methods: 1. Quantile tracking (P control). Running online gradient descent on the quantile loss (summed over all past scores) gives rise to a method that we call quantile tracking, which achieves long-run coverage (2) under no assumptions exce…”

14. resultمدعوم

    The target for online conformal control is long-run miscoverage 1/T∑_{t=1}^T err_t=α+o(1) as T tends to infinity.

    [1] Conformal PID Control for Time Series Prediction, section 1 Introduction

    “under few or no assumptions, where o⁡(1)o(1) denotes a quantity that tends to zero as T→∞T\to\infty. We note that (2) is not probabilistic at all, and every theoretical statement we will make in this paper holds deterministically. Furthermore, going beyond (2), we also seek to de…”

15. limitationمدعوم

    Without assumptions on the data sequence, achieving coverage at each individual time requires prediction intervals of infinite size, so the sequential objective is long-run rather than pointwise coverage.

    [1] Conformal PID Control for Time Series Prediction, section 1 Introduction

    “The challenge in the sequential setting is as follows. We seek to invert the score function to construct a conformal prediction set, 𝒞t={y∈𝒴:st​(xt,y)≤qt},\mathcal{C}_{t}=\{y\in\mathcal{Y}:s_{t}(x_{t},y)\leq q_{t}\}, (1) where qtq_{t} is an estimated 1−α1-\alpha quantile for the …”

16. methodمدعوم

    ACI models distribution shift as a learning problem in a single parameter whose optimal value varies over time and is continuously re-estimated.

    [2] Adaptive Conformal Inference Under Distribution Shift, abstract arXiv:2106.00170v3

    “We develop methods for forming prediction sets in an online setting where the data generating distribution is allowed to vary over time in an unknown fashion. Our framework builds on ideas from conformal inference to provide a general wrapper that can be combined with any black b…”

17. resultمدعوم

    ACI achieves the target coverage frequency over long time intervals without assumptions on the data-generating distribution.

    [2] Adaptive Conformal Inference Under Distribution Shift, section 1 Introduction

    “While exchangeability is a common assumption, there are many real-world applications in which we do not expect the marginal distribution of (Xt,Yt)(X_{t},Y_{t}) to be stationary. For example, in finance and economics market behaviour can shift drastically in response to new legis…”

18. resultمدعوم

    When distribution shift is small and the prediction algorithm has a specified simple form, ACI additionally obtains approximate marginal coverage at most time steps.

    [2] Adaptive Conformal Inference Under Distribution Shift, section 1 Introduction

    “While exchangeability is a common assumption, there are many real-world applications in which we do not expect the marginal distribution of (Xt,Yt)(X_{t},Y_{t}) to be stationary. For example, in finance and economics market behaviour can shift drastically in response to new legis…”

19. methodمدعوم مع تحفظات

    An online calibration set consists of previously observed pairs and is used to determine how small a conformity score must be for a candidate response to be accepted.

    [2] Adaptive Conformal Inference Under Distribution Shift, section 1.1 Conformal inferencePassage says calibration set D_cal ⊆ {(X_r,Y_r)}_{1≤r≤t−1} is used to determine how small S(X_t,y) should be to accept y. It does not explicitly say 'previously observed pairs,' though that is clearly implied by the index r<t.

    “Suppose we are given a fitted regression model for predicting the value of YY from XX. Let yy be a candidate value for YtY_{t}. To determine if yy is a reasonable estimate of YtY_{t}, we define a conformity score S⁡(X,Y)S(X,Y) that measures how well the value yy conforms with the…”

20. methodمدعوم

    The fitted score quantile is defined as Q^(p)=inf{s:(1/|D_cal|)∑_{(X_r,Y_r)∈D_cal}1{S(X_r,Y_r)≤s}≥p}, and a candidate y is accepted when S(X_t,y)≤Q^(1−α).

    [2] Adaptive Conformal Inference Under Distribution Shift, section 1.1 Conformal inference

    “used to fit the regression model. Using this calibration set we define the fitted quantiles of the conformity scores to be Q^(p):=inf{s:(1|𝒟cal|∑(Xr,Yr)∈𝒟cal𝟙{S(Xr,Yr)≤s})≥p},\hat{Q}(p):=\inf\left\{s:\left(\frac{1}{|\mathcal{D}_{\text{cal}}|}\sum_{(X_{r},Y_{r})\in\mathcal{D}_{\te…”

21. resultمدعوم

    Under exchangeability with uniformly randomized tie breaking, the split-conformal prediction set has coverage ⌈|D_cal|(1−α)⌉/(|D_cal|+1).

    [2] Adaptive Conformal Inference Under Distribution Shift, section 1.1 Conformal inference

    “used to fit the regression model. Using this calibration set we define the fitted quantiles of the conformity scores to be Q^(p):=inf{s:(1|𝒟cal|∑(Xr,Yr)∈𝒟cal𝟙{S(Xr,Yr)≤s})≥p},\hat{Q}(p):=\inf\left\{s:\left(\frac{1}{|\mathcal{D}_{\text{cal}}|}\sum_{(X_{r},Y_{r})\in\mathcal{D}_{\te…”

22. limitationمدعوم

    The usual conformal marginal-coverage guarantee requires exchangeability of the training and test data.

    [2] Adaptive Conformal Inference Under Distribution Shift, section 1 Introduction

    “Perhaps the most powerful and flexible tools for solving this problem come from conformal inference [34, 16, 32, 22, 31, 15, 3, see e.g.] . This framework provides a generic methodology for transforming the outputs of any black box prediction algorithm into a prediction set. The …”

23. resultمدعوم

    Bellman Conformal Inference provides approximately calibrated prediction intervals for time-series forecasting and achieves long-term coverage under arbitrary distribution shifts and temporal dependence.

    [4] Bellman Conformal Inference: Calibrating Prediction Intervals For Time Series, abstract S2 fe017333ffc7

    “We introduce Bellman Conformal Inference (BCI), a framework that wraps around any time series forecasting models and provides approximately calibrated prediction intervals. Unlike existing methods, BCI is able to leverage multi-step ahead forecasts and explicitly optimize the ave…”

24. resultمدعوم

    Bellman Conformal Inference empirically produces substantially shorter prediction intervals than existing methods and avoids uninformative intervals with infinite lengths.

    [4] Bellman Conformal Inference: Calibrating Prediction Intervals For Time Series, abstract S2 fe017333ffc7

    “We introduce Bellman Conformal Inference (BCI), a framework that wraps around any time series forecasting models and provides approximately calibrated prediction intervals. Unlike existing methods, BCI is able to leverage multi-step ahead forecasts and explicitly optimize the ave…”

25. uncertaintyمدعوم

    The provided passages report experiments on market returns but do not give coverage or interval-width numbers for financial OHLCV data at 5-minute through daily horizons, nor do they compare such numbers with raw foundation-model quantiles.

    [1] Conformal PID Control for Time Series Prediction, abstract arXiv:2307.16895v1

    “We study the problem of uncertainty quantification for time series prediction, with the goal of providing easy-to-use algorithms with formal guarantees. The algorithms we present build upon ideas from conformal prediction and control theory, are able to prospectively model confor…”

26. uncertaintyمدعوم

    The provided passages do not specify an implementation of weighted conformal prediction for streaming price data.

    [1] Conformal PID Control for Time Series Prediction, section 1.1 Peek at results: methods

    “Three design principles underlie our methods: 1. Quantile tracking (P control). Running online gradient descent on the quantile loss (summed over all past scores) gives rise to a method that we call quantile tracking, which achieves long-run coverage (2) under no assumptions exce…”

27. factمدعوم

    Conformal prediction provides valid predictive inference under exchangeability, but distribution drift violates exchangeability in deployed settings.

    [8] Conformal prediction beyond exchangeability, abstract DOI 10.1214/23-aos2276

    “Conformal prediction is a popular, modern technique for providing valid predictive inference for arbitrary machine learning models. Its validity relies on the assumptions of exchangeability of the data, and symmetry of the given model fitting algorithm as a function of the data. …”

28. methodمدعوم

    Weighted conformal prediction uses weighted quantiles to improve robustness against distribution drift.

    [8] Conformal prediction beyond exchangeability, abstract DOI 10.1214/23-aos2276

    “Conformal prediction is a popular, modern technique for providing valid predictive inference for arbitrary machine learning models. Its validity relies on the assumptions of exchangeability of the data, and symmetry of the given model fitting algorithm as a function of the data. …”

29. methodمدعوم

    The weighted conformal framework introduces a randomization technique that permits algorithms that do not treat data points symmetrically.

    [8] Conformal prediction beyond exchangeability, abstract DOI 10.1214/23-aos2276

    “Conformal prediction is a popular, modern technique for providing valid predictive inference for arbitrary machine learning models. Its validity relies on the assumptions of exchangeability of the data, and symmetry of the given model fitting algorithm as a function of the data. …”

30. resultمدعوم

    Under exchangeability, the weighted conformal methods achieve the same coverage guarantees as existing conformal prediction methods.

    [8] Conformal prediction beyond exchangeability, abstract DOI 10.1214/23-aos2276

    “Conformal prediction is a popular, modern technique for providing valid predictive inference for arbitrary machine learning models. Its validity relies on the assumptions of exchangeability of the data, and symmetry of the given model fitting algorithm as a function of the data. …”

31. resultمدعوم

    When exchangeability is violated by distribution drift or other challenging features of real data, the weighted conformal methods exhibit substantially less loss of coverage than existing methods.

    [8] Conformal prediction beyond exchangeability, abstract DOI 10.1214/23-aos2276

    “Conformal prediction is a popular, modern technique for providing valid predictive inference for arbitrary machine learning models. Its validity relies on the assumptions of exchangeability of the data, and symmetry of the given model fitting algorithm as a function of the data. …”

32. methodمدعوم

    The industrial time-series uncertainty framework constructs a prediction interval from two one-sided intervals.

    [5] Uncertainty Quantification Based on Conformal Prediction for Industrial Time Series With Distribution Shift, abstract S2 816973db1f62

    “Conformal prediction (CP) is known to theoretically guarantee prediction interval coverage under the exchangeability assumption. However, industrial time series collected from real-world industrial processes often violates this assumption due to temporal dependencies and distribu…”

33. methodمدعوم

    The framework combines entire and local nonconformity-score information to adjust the confidence levels of two one-tailed intervals over time.

    [5] Uncertainty Quantification Based on Conformal Prediction for Industrial Time Series With Distribution Shift, abstract S2 816973db1f62

    “Conformal prediction (CP) is known to theoretically guarantee prediction interval coverage under the exchangeability assumption. However, industrial time series collected from real-world industrial processes often violates this assumption due to temporal dependencies and distribu…”

34. resultمرفوض

    At a nominal confidence level of 95%, the industrial time-series framework achieved average empirical coverage of 95.0% and reduced prediction-interval width by 6.29% on the wastewater dataset compared with the best-performing benchmark model.

    [5] Uncertainty Quantification Based on Conformal Prediction for Industrial Time Series With Distribution Shift, abstract S2 816973db1f62

    “Conformal prediction (CP) is known to theoretically guarantee prediction interval coverage under the exchangeability assumption. However, industrial time series collected from real-world industrial processes often violates this assumption due to temporal dependencies and distribu…”

35. resultمدعوم

    On the actual sintering production dataset, the industrial time-series framework achieved 95.6% coverage and reduced prediction-interval width by 15.70% compared with the best-performing benchmark model.

    [5] Uncertainty Quantification Based on Conformal Prediction for Industrial Time Series With Distribution Shift, abstract S2 816973db1f62

    “Conformal prediction (CP) is known to theoretically guarantee prediction interval coverage under the exchangeability assumption. However, industrial time series collected from real-world industrial processes often violates this assumption due to temporal dependencies and distribu…”

36. methodمدعوم

    Ensemble conformalized quantile regression (EnCQR) applies conformal calibration on top of a generic forecasting model, including deep learning architectures.

    [9] Ensemble Conformalized Quantile Regression for Probabilistic Time Series Forecasting, abstract arXiv:2202.08756v2

    “This paper presents a novel probabilistic forecasting method called ensemble conformalized quantile regression (EnCQR). EnCQR constructs distribution-free and approximately marginally valid prediction intervals (PIs), which are suitable for nonstationary and heteroscedastic time …”

37. methodمدعوم

    EnCQR uses a bootstrap ensemble estimator to apply conformal predictors to time series without requiring data exchangeability.

    [9] Ensemble Conformalized Quantile Regression for Probabilistic Time Series Forecasting, abstract arXiv:2202.08756v2

    “This paper presents a novel probabilistic forecasting method called ensemble conformalized quantile regression (EnCQR). EnCQR constructs distribution-free and approximately marginally valid prediction intervals (PIs), which are suitable for nonstationary and heteroscedastic time …”

38. methodمدعوم

    EnCQR uses ensemble learners implemented as generic quantile-regression algorithms so that prediction-interval length adapts to local variability.

    [9] Ensemble Conformalized Quantile Regression for Probabilistic Time Series Forecasting, abstract arXiv:2202.08756v2

    “This paper presents a novel probabilistic forecasting method called ensemble conformalized quantile regression (EnCQR). EnCQR constructs distribution-free and approximately marginally valid prediction intervals (PIs), which are suitable for nonstationary and heteroscedastic time …”

39. resultمدعوم

    Experiments reported that EnCQR produced sharper, more informative, and valid prediction intervals than methods based only on quantile regression or only on conformal prediction.

    [9] Ensemble Conformalized Quantile Regression for Probabilistic Time Series Forecasting, abstract arXiv:2202.08756v2

    “This paper presents a novel probabilistic forecasting method called ensemble conformalized quantile regression (EnCQR). EnCQR constructs distribution-free and approximately marginally valid prediction intervals (PIs), which are suitable for nonstationary and heteroscedastic time …”

40. methodمدعوم

    EnbPI wraps around ensemble predictors, avoids data splitting and retraining, and is designed to produce sequential prediction intervals efficiently.

    [13] Conformal prediction for time series, section Conformal prediction for time series

    “Chen Xu Note: cxu310@gatech.edu Affiliation: H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology. Yao Xie Note: yao.xie@isye.gatech.edu Affiliation: H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Te…”