Signal temporal logic control synthesis among uncontrollable dynamic agents with conformal prediction

Hybrid conformal prediction framework for reliable seismic failure mode identification in reinforced concrete columns

Predictive inference under a general regression setting is gaining more interest in the big-data era. In terms of going beyond point prediction to develop prediction intervals, two main threads of development are conformal prediction and Model-free prediction. Recently, a new conformal prediction approach was proposed that exploits the same uniformization procedure as in the well-known Model-free Bootstrap. Hence, it is of interest to compare and further investigate the performance of the two methods. In the paper at hand, we contrast the two approaches via theoretical analysis and numerical experiments with a focus on conditional coverage of prediction intervals. We discuss suitable scenarios for applying each algorithm, underscore the importance of conditional vs. unconditional coverage, and show that, under mild conditions, the Model-free bootstrap yields prediction intervals with guaranteed better conditional coverage compared to quantile estimation. We also extend the concept of ‘pertinence’ of prediction intervals to the nonparametric regression setting, and give concrete examples where its importance emerges under finite sample scenarios. Finally, we define the new notion of ‘conjecture testing’ that is the analog of hypothesis testing as applied to the prediction problem; we also devise a modified conformal score to allow conformal prediction to handle one-sided ‘conjecture tests’, and compare to the Model-free bootstrap.

In recent years, many foundation generative models have been developed to predict structures of molecules and materials. Although these foundation models have achieved great success, it is challenging to collect enough data to train foundation generative models. One such example is to predict protein conformations with protein-environment interactions (PEIs), such as interactions introduced by organic linkers or material surfaces. We propose a physics-guided route to extrapolate foundation models beyond their training domain. Our method couples a pretrained deep generative model with explicit, physics-based interaction potentials for PEIs, steering sampling toward conformations consistent with external constraints without any retraining or fine-tuning. We demonstrate accurate and efficient conformation prediction of (i) cyclic peptide with organic linkers and (ii) peptide adsorbed on the gold surface. The generated structures serve as high-quality initial conditions for downstream simulations, providing a general, systematic approach to extend foundation models to proteins under system-specific environmental interactions.

Objectives— We establish and rigorously evaluate an Android malware detector that optimizes not only discrimination (how well samples are ranked) but also reliability: (i) probabilities that correspond to empirical risks and (ii) explicit, distribution free uncertainty quantification. Our goals are to (1) benchmark strong tabular baselines on a canonical CSV corpus; (2) calibrate scores so a value like 0.9 aligns with “roughly 90% malicious”; and (3) add inductive conformal prediction (ICP) to provide finite-sample coverage guarantees and a principled abstain option. Methodology— The pipeline: (i) ingest and sanitize a dataset of 15,036 applications with static features; (ii) encode mixed datatypes via one-hot and factorization; (iii) train Logistic Regression (LR), Random Forest (RF), and XGBoost (GBDT); (iv) report ROC–AUC, PR–AUC, F1, Accuracy, Brier score; (v) apply post hoc calibration (Platt, Isotonic) and compute Expected Calibration Error (ECE) with reliability diagrams including Wilson intervals; and (vi) deploy ICP to return prediction sets achieving target error α ∈ {0.10,0.05,0.01}, yielding selective automation. Dataset— A text-only (CSV) Android corpus of static attributes (manifest/permissions/API-like proxies) with a binary label (benign/malicious). Preprocessing drops identifiers, coerces types, imputes missing values, applies de-duplication, and uses an 80/20 stratified split with a 20% calibration carve-out from training (effective 64/16/20 train/cal/test). Results— RF/GBDT provide the best ranking. Calibration consistently lowers ECE/Brier and aligns reliability curves with the identity line; we visualize the effect for RF. ICP tracks the nominal target coverage closely with average set size near one, indicating selective abstentions. We include a temporal slice analysis to illustrate robustness under modest distribution shift, plus an operational thresholding example linking (α,t) to analyst load. Applications— Calibrated probabilities support auditable thresholds in app stores and enterprise gateways; ICP provides a knob for predictable abstentions under finite-sample guarantees. The approach is model-agnostic, simple to reproduce, and easy to retrofit in static or hybrid pipelines. Availability— Scripts for preprocessing, calibration, ICP, metric computation, and figure generation, along with CSV/PNG artifacts referenced in the paper, are provided in the artifact bundle.

Bacterial antimicrobial resistance is one of the most pressing global health challenges. Infections with resistant pathogens increase patient morbidity and mortality due to limited treatment options. Rapid and reliable identification of resistance is therefore crucial. However, conventional culture-based diagnostics are slow, typically requiring at least 48 hours from patient sample arrival to result. In contrast, matrix-assisted laser desorption/ionization time-of-flight (MALDI-TOF) mass spectrometry, routinely used for species identification, can provide data 24 hours earlier. Repurposing this technique for antimicrobial resistance prediction has shown promise, but limited predictive performance and a lack of statistically grounded uncertainty estimates have hindered clinical integration. To address these issues, we propose an antimicrobial resistance detection framework using a knowledge-graph-enhanced conformal predictor. Conformal prediction outputs sets of likely effective antibiotics with statistical guarantees, ensuring that resistance detection meets a predefined error rate. Our approach improves upon standard conformal prediction by integrating domain knowledge through a drug- and species-specific knowledge graph that captures interdependencies between antibiotics, such as inferable resistance patterns between broad- and narrow-spectrum agents, as well as co-resistance patterns within antibiotic classes. This predictor is layered on top of a novel classifier that surpasses state-of-the-art models and overcomes key technical limitations of earlier approaches. We evaluated our framework on two clinically relevant species, Klebsiella pneumoniae and Escherichia coli, using the DRIAMS dataset. Our results demonstrate that our conformal predictor consistently achieved the expected coverage guarantees and that the knowledge-graph enhancement significantly reduced false discovery rates compared to standard conformal approaches. By adding statistically grounded uncertainty estimates and improving predictive performance, our framework strengthens the reliability of early antimicrobial resistance predictions from MALDI-TOF data. This could support the clinical integration of such rapid diagnostics by increasing trust in their results and enabling better-informed early treatment decisions.1.

Integrating deep learning and mobile imaging for assessment of automated conformational indices and weight prediction in Brahman cattle

Satellite pose estimation constitutes a critical technology in the aerospace tasks. The tradeoff between accuracy and efficiency becomes paramount for successful mission execution, due to the limited computational resources of on-board systems. Existing methods predominantly provide single-point estimations, which fall short of fulfilling the uncertainty quantification requirements demanded by safety-critical space operations. To address these problems, we first propose uncertainty-guided conformal keypoint detection to predict keypoint inductive conformal prediction (IndCP) set and then design a uncertainty propagation strategy to obtain pose uncertainty set. Specifically, we build our method upon a transformer-based keypoint predictor, which directly outputs uncertainty-guided keypoints. We first propose a nonconformal function to generate keypoint IndCP set to cover the ground-truth keypoint with a certain probability. We then apply Monte Carlo to sample within the keypoint IndCP set and estimate the poses by solving the perspective-n-point (PnP) problem. The top-n poses with the smallest conformal reprojection error are used to construct a convex hull, which are defined as the pose uncertainty set. Furthermore, we take the mean of the top-n poses as the average pose. Experiments on the Spacecraft PosE Estimation challenge Dataset (SPEED) and LineMOD Occlusion (LMO) dataset show that not only the average pose demonstrates higher accuracy but also the pose uncertainty sets can cover the true pose with the certain probability.

Dual-splitting conformal prediction for multi-step time series forecasting

Understanding and explaining complex geographic phenomena—ranging from climate change to socioeconomic disparities—is a central focus in both geography and the broader scientific community. Various methods have been developed to elucidate relationships between variables, from coefficient estimates in linear regression models to the increasingly dominant use of feature attribution scores in Explainable AI (XAI) techniques. However, explanations generated by XAI methods often carry uncertainty, stemming from the model itself and the data used to train the model. Despite the critical importance of accounting for such uncertainty, this issue remains largely overlooked in the geospatial domain. In this study, we developed an uncertainty quantification framework for XAI explanations based on conformal prediction, termed Geospatial eXplanation Conformal Prediction (GeoXCP). By incorporating spatial dependence into the modeling process, GeoXCP produced spatially adaptive explanations with calibrated uncertainty estimates. We validated the effectiveness of GeoXCP through extensive simulation experiments and real-world datasets. The results demonstrated that GeoXCP provided reliable explanations while effectively quantifying uncertainty across diverse geospatial scenarios. Our approach represented a significant advancement in explainable geospatial machine learning, enabling decision-makers to better assess the trustworthiness of model-driven insights. The proposed framework was implemented in a python package, named GeoXCP.

While Electronic Health Records (EHRs) promise comprehensive documentation of patient care, in reality there are significant challenges in data reliability and utilization. EHRs contain vast amounts of unstructured clinical narratives that, despite containing critical and relevant medical information, remain difficult to systematically extract and verify. Recent advances in large language models (LLMs) offer increasingly improving capabilities for extracting structured information from clinical notes, yet these approaches raise fundamental questions about output reliability, over-confident token predictions, and provide no guarantees (statistical or otherwise) for downstream clinical applications. In this work, we present a conformal verification framework for unstructured EHR data extraction using generative AI. While LLMs have increasingly impressive capabilities, they are notoriously miscalibrated and overconfident in their predictions, necessitating rigorous verification methods to eliminate the need to trust AI models. Our approach (i) employs LLMs to extract medical entities and concepts from clinical narratives with LLM-as-a-judge verification, (ii) implements probabilistic calibration to quantify extraction confidence, and (iii) applies conformal prediction to provide finite-sample guarantees on error rates for accepted extractions. We evaluate our framework on 10k clinical visits across 898 clinical practices utilizing three different EHR systems. Our conformal verification approach can provide assurances that the future expected proportion of accepted but incorrect extractions remains below a pre-specified risk level with rigorous statistical verification. It also maintains formal guarantees over clinical data quality, and illuminates the miscalibrations present in state-of-the-art LLM models, requiring additional validation for safe deployment of automated extraction systems.

Accurate estimation of core body temperature (CBT) is essential for physiological monitoring, yet current non-invasive methods lack statistically calibrated uncertainty estimates required for safety-critical use. Here we introduce a conformal deep learning framework for real-time, non-invasive CBT prediction with calibrated uncertainty, demonstrated in high-risk heat-stress environments. Developed from over 140,000 physiological measurements across six operational domains, the model achieves a test error of 0.29 °C, outperforming the widely used ECTemp™ algorithm with a 12-fold improvement in calibrated probabilistic accuracy and statistically valid prediction intervals. Designed for integration with wearable devices, the system uses accessible physiological, demographic, and environmental inputs to support practical, confidence-informed monitoring. A customizable alert engine enables proactive safety interventions based on user-defined thresholds and model confidence. By combining deep learning with conformal prediction, this approach establishes a generalizable foundation for trustworthy, non-invasive physiological monitoring, demonstrated here for CBT under heat stress but applicable to broader safety-critical settings.

Accurate and reliable prediction has profound implications for a wide range of applications, such as hospital admissions, inventory control, and route planning. In this study, we focus on an instance of spatiotemporal learning problems—traffic prediction—to demonstrate an advanced deep learning model developed for making accurate and reliable predictions. Despite the significant progress in traffic prediction, limited studies have incorporated both explicit (e.g., road network topology) and implicit (e.g., causality-related traffic phenomena and impact of exogenous factors) traffic patterns simultaneously to improve prediction performance. Meanwhile, the variable nature of traffic states necessitates quantifying the uncertainty of model predictions in a statistically principled way; however, extant studies offer no provable guarantee on the statistical validity of confidence intervals in reflecting their actual likelihood of containing the ground truth. In this paper, we propose an end-to-end traffic prediction framework that leverages three primary components to generate accurate and reliable traffic predictions: dynamic causal structure learning for discovering implicit traffic patterns from massive traffic data, causally aware spatiotemporal multigraph convolutional network (CASTMGCN) for learning spatiotemporal dependencies, and conformal prediction for uncertainty quantification. In particular, CASTMGCN fuses several graphs that characterize different important aspects of traffic networks (including physical road structure, time-lagged causal effect, and contemporaneous causal relationships) and an auxiliary graph that captures the effect of exogenous factors on the road network. On this basis, a conformal prediction approach tailored to spatiotemporal data is further developed for quantifying the uncertainty in node-wise traffic predictions over varying prediction horizons. Experimental results on two real-world traffic data sets of varying scale demonstrate that the proposed method outperforms several state-of-the-art models in prediction accuracy; moreover, it generates more efficient prediction regions than several other methods while strictly satisfying the statistical validity in coverage. History: Accepted by Ram Ramesh, Area Editor for Data Science and Machine Learning. Funding: This paper was supported by the Hong Kong Research Grants Council [Grant PolyU 25206422], the Research Committee of The Hong Kong Polytechnic University [Project Code G-UARJ, Student Account Code RM5Y], and the National Natural Science Foundation of China [Grants 62406269, 72021002, and 72201180]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2024.0891 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2024.0891 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Fair conformal prediction for incomplete covariate data

Atmospheric pollution causes millions of excess deaths annually, with particulate matter (PM) being a major concern. While research has traditionally focused on PM10 and PM2.5, ultrafine particles (UFPs, diameter < 100 nm) have emerged as a critical human health risk due to their ability to penetrate deeply into the respiratory system, transmigrate into the bloodstream and induce systemic health impacts. The total particle number concentration (PNC) serves as a proxy measure for UFP prevalence, as UFPs dominate particle number counts despite contributing minimally to total particle mass. This study presents the first global datasets of PNCs and UFPs at 1 km resolution over land by combining ground station measurements with machine learning. We developed an XGBoost model to predict annual PNC levels from 2010–2019, integrating diverse environmental and anthropogenic variables available at the global scale. Our model achieves an R2 of ≥0.9 and a mean relative error of about 30% for polluted urban areas, based on comparison with test datasets, and its performance was evaluated by including spatial and temporal cross-validation schemes. We find that global annual mean PNCs near the Earth’s surface vary between a few thousand per cm3 in pristine environments up to more than 40,000 per cm3 in some urban centres and that UFPs contribute about 91% to PNCs. The model incorporates a conformal prediction framework to provide reliable coverage intervals, making local-to-global PNC and UFP data available and supporting exposure assessments and health impact studies.

Deep neural networks have changed several disciplines and have reached levels of accuracy never seen before in difficult jobs like analysing medical images and perceiving autonomous vehicles. However, even though current neural networks are quite good at classifying things, they typically have bad calibration, which leads to predictions that are too confident and don't show how uncertain the predictions really are. This seriously hurts the dependability and trustworthiness of the system. This study offers a thorough and detailed examination of cutting-edge calibration and uncertainty quantification techniques aimed at guaranteeing dependable confidence assessments in safety-critical systems. We systematically survey fundamental concepts of model calibration, comprehensively examine the root causes of miscalibration in modern deep neural networks, thoroughly discuss calibration methods including post-hoc techniques such as temperature scaling, Platt scaling, and isotonic regression, regularisation approaches including label smoothing, mixup, and focal loss, and sophisticated uncertainty estimation frameworks including Bayesian neural networks, Monte Carlo dropout, deep ensembles, and conformal prediction. We provide an in-depth examination of assessment metrics used to gauge calibration quality, such as ECE, MCE, Brier score, and log loss. We also investigate their practical applications in the fields of medical imaging and autonomous driving, while pinpointing existing obstacles and prospective research avenues. The results show that there are several efficient calibration solutions that work well together. For example, temperature scaling makes things better without adding much to the cost of computing, ensemble approaches work well, and conformal prediction gives theoretical assurances.

Modeling the operational dynamics of intricate rail transit systems faces three significant challenges: addressing network-wide dependencies, differentiating correlation from causation, and accurately quantifying prediction uncertainty. Current methodologies generally tackle these issues separately, resulting in models that are either structurally simplistic, causally unclear, or overly confident in their forecasts. This paper presents a comprehensive framework that, for the first time, effectively combines Graph Neural Networks (GNNs), Causal Machine Learning (CML), and Conformal Prediction (CP) to resolve this dilemma. GNNs are utilized to capture the topological dependencies within the rail network, CML is employed to discern the unbiased causal impacts of operational interventions, and CP offers mathematically assured, distribution-free uncertainty intervals. Our empirical assessment using real-world operational data reveals a distinct differentiation in model performance: while GNN-enhanced hybrids excel in aggregate prediction tasks (CV R² ≈ 0.87), the proposed CML-CP framework realizes a transformative, order-of-magnitude decrease in causal effect estimation error (CV MAE: 124,758.04). Thus, the primary contribution of this research is not merely a singular "best" model, but rather a methodological roadmap that facilitates a paradigm shift from reactive data modeling to a proactive, strategic decision-support tool. This framework equips decision-makers to perform reliable what-if scenario analyses, supported by robust causal insights and valid uncertainty guarantees, leading to more resilient and efficient railway operations.

Rapid advances in perception have enabled large pre-trained models to be used out of the box for transforming high-dimensional, noisy, and partial observations of the world into rich occupancy representations. However, the reliability of these models and consequently their safe integration onto robots remains unknown, particularly when deployed in environments unseen during training. To provide safety guarantees, we rigorously quantify the uncertainty of pre-trained perception systems for object detection and scene completion via a novel calibration technique based on conformal prediction. Crucially, this procedure guarantees robustness to distribution shifts in states when perception outputs are used in conjunction with a planner. As a result, the calibrated perception system can be used in combination with any safe planner to provide an end-to-end statistical assurance on safety in unseen environments. We evaluate the resulting approach, Perceive with Confidence ( PwC ), in simulation and on hardware where a quadruped robot navigates through previously unseen static indoor environments. These experiments validate the safety assurances for obstacle avoidance provided by PwC . In our simulation experiments, our method reduces obstacle misdetection significantly compared to uncalibrated perception models. While misdetections lead to collisions for baseline methods, our approach remains safe. We further demonstrate reducing the conservatism of our method without sacrificing safety, outperforming all baselines in success rates in challenging environments. In hardware experiments on a quadruped robot, our method improves empirical safety and obstacle misdetection by significant margins over the baselines, highlighting our approach’s robustness under more demanding conditions.

Advancing risk prediction in sickle cell disease through multi-task deep learning of mortality and intermediate risk factors

Conformal predictors in chemometric study of mid-infrared food adulteration: quantification of prediction uncertainty.