Individualized conformal prediction: using synthetic data as relevant controls

Conformalized quantile regression, a cutting-edge and model-agnostic algorithm, has emerged as a recent innovation to generate valid prediction intervals on finite samples while addressing heteroscedasticity. It starts by employing quantile regression to estimate conditional quantiles. Subsequently, these estimated conditional quantiles undergo a rectification process using conformal prediction. Under the assumption of exchangeability, a slightly weaker form of independent and identically distributed (i.i.d.) data, the resulting prediction intervals are valid in finite samples. However, a drawback of the proposed conformalization step is identified: it lacks the capacity to adapt to heteroscedasticity due to its independence from the input. To overcome this limitation, we propose an improvement that involves partitioning the covariates space into clusters, assigning higher weights to features with greater predictive power. Following that, within each cluster, a conformal step is applied, leveraging a rectification that is reliant on the input cluster-wise.To demonstrate the superiority of our improved version over the classic version of conformalized quantile regression, we conducted a comprehensive comparison of their respective prediction intervals using synthetic data.

In this paper, we develop an inferential method based on conformal prediction, which can wrap around any survival prediction algorithm to produce calibrated, covariate-dependent lower predictive bounds on survival times. In the Type I right-censoring setting, when the censoring times are completely exogenous, the lower predictive bounds have guaranteed coverage in finite samples without any assumptions other than that of operating on independent and identically distributed data points. Under a more general conditionally independent censoring assumption, the bounds satisfy a doubly robust property which states the following: marginal coverage is approximately guaranteed if either the censoring mechanism or the conditional survival function is estimated well. The validity and efficiency of our procedure are demonstrated on synthetic data and real COVID-19 data from the UK Biobank.

With the development of science and technology and the explosive growth of data, there will be a lot of trajectories every day. However, how to detect the abnormal trajectory from many trajectories has become a hot issue. In order to study trajectory anomaly detection better, we analyze the Sequential conformal anomaly detection in trajectories based on hausdorff distance (SNN-CAD) method, and propose a new measurement method of trajectory distance Improved Moved Euclidean Distance (IMED) instead of Hausdorff distance, which reduces the computational complexity. In addition, we propose a removing-updating strategy to enhance the conformal prediction (CP). Then, we also put forward our Non-conformity measure (NCM), Mean Distance Deviation. It can enlarge the difference between trajectories more effectively, and detect the abnormal trajectory more accurately. Finally, based on the technical measures mentioned above and under the framework of enhanced conformal prediction theory detection, we also build our own detector called Mean Distance Deviation Detector (MDD-ECAD). Using a large number of synthetic trajectory data and real world trajectory data on two detectors, the experimental results show that MDD-ECAD is much better than SNN-CAD in both accuracy and running time.KeywordsConformal predictionNon-conformity measureTrajectory distance

Abstract: We develop a causal time-series framework using statsmodels to maintain calibrated predictive risk under regime shifts. Combining ARIMAX/VAR with drift detection and conformal risk control (CRC), we adapt the risk level α_t online to preserve target coverage while keeping intervals compact. On synthetic regime-switching data, CRC with change-point signals recovers nominal 90% coverage with 3–8% tighter intervals versus static conformal baselines and reduces false alerts. We present a conceptual model, mathematical formulation, and evaluation protocol aligned with regulatory expectations for model risk management. Keywords: Causal Time-Series; statsmodels; ARIMAX; VAR; Conformal Prediction; Risk Control; Change-Point Detection; Regime Shifts.

Modern advances in machine learning (ML) and wearable medical sensors (WMSs) have enabled out-of-clinic disease detection. However, a trained ML model often suffers from misclassification when encountering non-stationary data domains after deployment. Use of continual learning (CL) strategies is a common way to perform domain-incremental adaptation while mitigating catastrophic forgetting. Nevertheless, most existing CL methods require access to previously learned domains through preservation of raw training data or distilled information. This is often infeasible in real-world scenarios due to storage limitations or data privacy, especially in smart healthcare applications. Moreover, it makes most existing CL algorithms inapplicable to deployed models in the field, thus incurring re-engineering costs. To address these challenges, we propose PAGE, a domain-incremental adaptation strategy with past-agnostic generative replay for smart healthcare. PAGE enables generative replay without the aid of any preserved data or information from prior domains. When adapting to a new domain, it exploits real data from the new distribution and the current model to generate synthetic data that retain the learned knowledge of previous domains. By replaying the synthetic data with the new real data during training, PAGE achieves a good balance between domain adaptation and knowledge retention. In addition, we incorporate an extended inductive conformal prediction (EICP) method into PAGE to produce a confidence score and a credibility value for each detection result. This makes the predictions interpretable and provides statistical guarantees for disease detection in smart healthcare applications. We demonstrate PAGE’s effectiveness in domain-incremental disease detection with three distinct disease datasets collected from commercially available WMSs. PAGE achieves highly competitive performance against state-of-the-art along with superior scalability, data privacy, and feasibility. Furthermore, PAGE is able to enable up to 75% reduction in clinical workload with the help of EICP.

Intelligent Transportation Systems employ different localization technologies, such as the Global Navigation Satellite System. This system transmits signals between satellite and receiver devices on the ground which can estimate their position on earth’s surface. The accuracy of this positioning estimate, or the positioning error estimation, is of utmost importance for the efficient and safe operation of autonomous vehicles, which require not only the position estimate, but also an estimation of their operation margin. This paper proposes a workflow for positioning error estimation using a random forest regressor along with a post-hoc conformal prediction framework. The latter is calibrated on the random forest out-of-bag samples to transform the obtained positioning error estimates into predicted integrity intervals, which are confidence intervals on the positioning error prediction with at least 99.999\(\%\) confidence. The performance is measured as the number of ground truth positioning errors inside the predicted integrity intervals. An extensive experimental evaluation is performed on real-world and synthetic data in terms of root mean square error between predicted and ground truth positioning errors. Our solution results in an improvement of 73\(\%\) compared to earlier research, while providing prediction statistical guarantees.

SummaryConformal prediction is a general method that converts almost any point predictor to a prediction set. The resulting set retains the good statistical properties of the original estimator under standard assumptions, and guarantees valid average coverage even when the model is mis-specified. A main challenge in applying conformal prediction in modern applications is efficient computation, as it generally requires an exhaustive search over the entire output space. In this paper we develop an exact and computationally efficient conformalization of the lasso and elastic net. The method makes use of a novel piecewise linear homotopy of the lasso solution under perturbation of a single input sample point. As a by-product, we provide a simpler and better-justified online lasso algorithm, which may be of independent interest. Our derivation also reveals an interesting accuracy-stability trade-off in conformal inference, which is analogous to the bias-variance trade-off in traditional parameter estimation. The practical performance of the new algorithm is demonstrated in both synthetic and real data examples.

A conformal prediction inspired approach for distribution regression with random Fourier features

Pattern recognition is becoming an increasingly important tool for making inferences from the massive amounts of data produced in magnetic confinement fusion experiments. However, the measurements obtained from the various plasma diagnostics are typically affected by a considerable statistical uncertainty. In this work, we consider the inherent stochastic nature of the data by modeling the measurements by probability distributions in a metric space. Information geometry permits the calculation of the geodesic distances on such manifolds, which we apply to the important problem of the classification of plasma confinement regimes. We use a distance-based conformal predictor, which we first apply to a synthetic data set. Next, the method yields an excellent classification performance with measurements from an international database. The conformal predictor also returns confidence and credibility measures, which are particularly important for interpretation of pattern recognition results in stochastic fusion data.

To perform anomaly detection for trajectory data, we study the Sequential Hausdorff Nearest-Neighbor Conformal Anomaly Detector (SHNN-CAD) approach, and propose an enhanced version called SHNN-CAD. SHNN-CAD was introduced based on the theory of conformal prediction dealing with the problem of online detection. Unlike most related approaches requiring several not intuitive parameters, SHNN-CAD has the advantage of being parameter-light which enables the easy reproduction of experiments. We propose to adaptively determine the anomaly threshold during the online detection procedure instead of predefining it without any prior knowledge, which makes the algorithm more usable in practical applications. We present a modified Hausdorff distance measure that takes into account the direction difference and also reduces the computational complexity. In addition, the anomaly detection is more flexible and accurate via a re-do strategy. Extensive experiments on both real-world and synthetic data show that SHNN-CAD outperforms SHNN-CAD with regard to accuracy and running time.