Inference Algorithm Research Articles

Approximate Gibbs sampler for efficient inference of hierarchical Bayesian models for grouped count data

Hierarchical Bayesian Poisson regression models (HBPRMs) provide a flexible modelling approach of the relationship between predictors and count response variables. The applications of HBPRMs to large-scale datasets require efficient inference algorithms due to the high computational cost of inferring many model parameters based on random sampling. Although Markov Chain Monte Carlo (MCMC) algorithms have been widely used for Bayesian inference, sampling using this class of algorithms is time-consuming for applications with large-scale data and time-sensitive decision-making, partially due to the non-conjugacy of many models. To overcome this limitation, this research develops an approximate Gibbs sampler (AGS) to efficiently learn the HBPRMs while maintaining the inference accuracy. In the proposed sampler, the data likelihood is approximated with Gaussian distribution such that the conditional posterior of the coefficients has a closed-form solution. Numerical experiments using real and synthetic datasets with small and large counts demonstrate the superior performance of AGS in comparison to the state-of-the-art sampling algorithm, especially for large datasets.

Integrating patients in time series clinical transcriptomics data.

Analysis of time series transcriptomics data from clinical trials is challenging. Such studies usually profile very few time points from several individuals with varying response patterns and dynamics. Current methods for these datasets are mainly based on linear, global orderings using visit times which do not account for the varying response rates and subgroups within a patient cohort. We developed a new method that utilizes multi-commodity flow algorithms for trajectory inference in large scale clinical studies. Recovered trajectories satisfy individual-based timing restrictions while integrating data from multiple patients. Testing the method on multiple drug datasets demonstrated an improved performance compared to prior approaches suggested for this task, while identifying novel disease subtypes that correspond to heterogeneous patient response patterns. The source code and instructions to download the data have been deposited on GitHub at https://github.com/euxhenh/Truffle.

Propensity Weighted federated learning for treatment effect estimation in distributed imbalanced environments

Estimating treatment effects from observational data in medicine using causal inference is a very relevant task due to the abundance of observational data and the ethical and cost implications of conducting randomized experiments or experimental interventions. However, how could we estimate the effect of a treatment in a hospital that has very restricted access to treatment? In this paper, we want to address the problem of distributed causal inference, where hospitals not only have different distributions of patients, but also different treatment assignment criteria. Furthermore, it is necessary to take into account that due to privacy restrictions, personal patient data cannot be shared between hospitals. To address this problem, we propose an adaptation of the federated learning algorithm FederatedAveraging to one of the most advanced models for the prediction of treatment effects based on neural networks, TEDVAE. Our algorithm adaptation takes into account the shift in the treatment distribution between hospitals and is therefore called Propensity WeightedFederatedAveraging (PW FedAvg). As the distributions of the assignment of treatments become more unbalanced between the nodes, the estimation of causal effects becomes more challenging. The experiments show that PW FedAvg manages to reduce errors in the estimation of individual causal effects when imbalances are large, compared to VanillaFedAvg and other federated learning-based causal inference algorithms based on the application of federated learning to linear parametric models, Gaussian Processes and Random Fourier Features.

Multimodal subtypes identified in Alzheimer's Disease Neuroimaging Initiative participants by missing-data-enabled subtype and stage inference.

Alzheimer's disease is a highly heterogeneous disease in which different biomarkers are dynamic over different windows of the decades-long pathophysiological processes, and potentially have distinct involvement in different subgroups. Subtype and Stage Inference is an unsupervised learning algorithm that disentangles the phenotypic heterogeneity and temporal progression of disease biomarkers, providing disease insight and quantitative estimates of individual subtype and stage. However, a key limitation of Subtype and Stage Inference is that it requires a complete set of biomarkers for each subject, reducing the number of datapoints available for model fitting and limiting applications of Subtype and Stage Inference to modalities that are widely collected, e.g. volumetric biomarkers derived from structural MRI. In this study, we adapted the Subtype and Stage Inference algorithm to handle missing data, enabling the application of Subtype and Stage Inference to multimodal data (magnetic resonance imaging, positron emission tomography, cerebrospinal fluid and cognitive tests) from 789 participants in the Alzheimer's Disease Neuroimaging Initiative. Missing-data Subtype and Stage Inference identified five subtypes having distinct progression patterns, which we describe by the earliest unique abnormality as 'Typical AD with Early Tau', 'Typical AD with Late Tau', 'Cortical', 'Cognitive' and 'Subcortical'. These new multimodal subtypes were differentially associated with age, years of education, Apolipoprotein E (APOE4) status, white matter hyperintensity burden and the rate of conversion from mild cognitive impairment to Alzheimer's disease, with the 'Cognitive' subtype showing the fastest clinical progression, and the 'Subcortical' subtype the slowest. Overall, we demonstrate that missing-data Subtype and Stage Inference reveals a finer landscape of Alzheimer's disease subtypes, each of which are associated with different risk factors. Missing-data Subtype and Stage Inference has broad utility, enabling the prediction of progression in a much wider set of individuals, rather than being restricted to those with complete data.

A New Fuzzy Bayesian Inference Approach for Risk Assessments

Bayesian network (BN) inference is an important statistical tool with additive symmetry. However, BN inference cannot deal with the uncertain, fuzzy, random, and conflicting information from experts’ knowledge in the process of conducting a risk assessment. To tackle this issue, a new fuzzy BN inference approach for risk assessments was proposed based on cloud model (CM), interval type-2 fuzzy set (IT2 FS), interval type-2 fuzzy logic system (IT2 FLS), modified Dempster–Shafer (D-S) evidence theory (ET), and Latin hypercube sampling (LHS) methods along the following lines. Firstly, CM was integrated into IT2 FS, and CM-based IT2 FS (CM-IT2 FS) was defined in the IT2 FLS. Secondly, modified D-S ET was utilized to determine the CM-IT2 FS-based a priori probabilities, and the CM-IT2 FS-based BN model was established. Thirdly, the CM-IT2 FS-based a priori probabilities were reduced to the CM-IT1 FS-based ones using a type reducer in the IT2 FLS, LHS was applied to propose a new fuzzy BN inference algorithm, and then, the new algorithm was used in a typical case to perform the fuzzy BN positive inference for risk prediction and the fuzzy BN reverse inference for risk sensitivity analysis. Finally, the BN inference results were analyzed using the proposed algorithm and the two common BN inference algorithms, and the effectiveness of the proposed approach was validated. It can be concluded that the proposed approach was both accurate and promising.

Adaptive Conformal Inference for Computing Market Risk Measures: An Analysis with Four Thousand Crypto-Assets

This paper investigates the estimation of the value at risk (VaR) across various probability levels for the log-returns of a comprehensive dataset comprising four thousand crypto-assets. Employing four recently introduced adaptive conformal inference (ACI) algorithms, we aim to provide robust uncertainty estimates crucial for effective risk management in financial markets. We contrast the performance of these ACI algorithms with that of traditional benchmark models, including GARCH models and daily range models. Despite the substantial volatility observed in the majority of crypto-assets, our findings indicate that ACI algorithms exhibit notable efficacy. In contrast, daily range models, and to a lesser extent, GARCH models, encounter challenges related to numerical convergence issues and structural breaks. Among the ACI algorithms, Fully Adaptive Conformal Inference (FACI) and Scale-Free Online Gradient Descent (SF-OGD) stand out for their ability to provide precise VaR estimates across all quantiles examined. Conversely, Aggregated Adaptive Conformal Inference (AgACI) and Strongly Adaptive Online Conformal Prediction (SAOCP) demonstrate proficiency in estimating VaR for extreme quantiles but tend to be overly conservative for higher probability levels. These conclusions withstand robustness checks encompassing the market capitalization of crypto-assets, time-series size, and different forecasting methods for asset log-returns. This study underscores the promise of ACI algorithms in enhancing risk assessment practices in the context of volatile and dynamic crypto-asset markets.

Lifting in Support of Privacy-Preserving Probabilistic Inference

AbstractPrivacy-preserving inference aims to avoid revealing identifying information about individuals during inference. Lifted probabilistic inference works with groups of indistinguishable individuals, which has the potential to prevent tracing back a query result to a particular individual in a group. Therefore, we investigate how lifting, by providing anonymity, can help preserve privacy in probabilistic inference. Specifically, we show correspondences between k-anonymity and lifting and present s-symmetry as an analogue as well as PAULI, a privacy-preserving inference algorithm that ensures s-symmetry during query answering.

A simple phylogenetic approach to analyze hypermutated HIV proviruses reveals insights into their dynamics and persistence during antiretroviral therapy.

Hypermutated proviruses, which arise in a single HIV replication cycle when host antiviral APOBEC3 proteins introduce extensive G-to-A mutations throughout the viral genome, persist in all people living with HIV receiving antiretroviral therapy (ART). But, the within-host evolutionary origins of hypermutated sequences are incompletely understood because phylogenetic inference algorithms, which assume that mutations gradually accumulate over generations, incorrectly reconstruct their ancestor-descendant relationships. Using > 1400 longitudinal single-genome-amplified HIV env-gp120 sequences isolated from six women over a median 18 years of follow-up - including plasma HIV RNA sequences collected over a median 9 years between seroconversion and ART initiation, and > 500 proviruses isolated over a median 9 years on ART - we evaluated three approaches for removing hypermutation from nucleotide alignments. Our goals were to 1) reconstruct accurate phylogenies that can be used for molecular dating and 2) phylogenetically infer the integration dates of hypermutated proviruses persisting during ART. Two of the tested approaches (stripping all positions containing putative APOBEC3 mutations from the alignment, or replacing individual putative APOBEC3 mutations in hypermutated sequences with the ambiguous base R) consistently normalized tree topologies, eliminated erroneous clustering of hypermutated proviruses, and brought env-intact and hypermutated proviruses into comparable ranges with respect to multiple tree-based metrics. Importantly, these corrected trees produced integration date estimates for env-intact proviruses that were highly concordant with those from benchmark trees that excluded hypermutated sequences, indicating that the corrected trees can be used for molecular dating. Use of these trees to infer the integration dates of hypermutated proviruses persisting during ART revealed that these spanned a wide age range, with the oldest ones dating to shortly after infection. This indicates that hypermutated proviruses, like other provirus types, begin to be seeded into the proviral pool immediately following infection, and can persist for decades. In two of the six participants, hypermutated proviruses differed from env-intact ones in terms of their age distributions, suggesting that different provirus types decay at heterogeneous rates in some hosts. These simple approaches to reconstruct hypermutated provirus' evolutionary histories, allow insights into their in vivo origins and longevity, towards a more comprehensive understanding of HIV persistence during ART.

Improving crop yield estimation by unified model parameters and state variable with Bayesian inference

Data assimilation techniques integrating crop growth models and remote sensing technologies offer a feasible approach for large-scale crop yield estimation. Previous research has primarily focused on either recalibrate the uncertain model parameters or updating model state variables independently using remotely sensed observations. In this study, we developed a two-step inference algorithm that couples the parameter inference and the state update, to solve the joint posterior distribution of uncertain parameters and state variables given the remote sensing observations. An Observing Simulation System (OSS) experiment was first performed based on the WOFOST crop model to validate the effectiveness of the parameter inference method. The results indicate that, the parameter inference method successfully improved the estimation of different types of model parameters and enhanced yield estimation. Furthermore, leaf area index (LAI) retrieved from Sentinel-2 was assimilated into the WOFOST model to simulate winter wheat yield at the plot scale in the northeastern part of Henan Province. The results demonstrated that the proposed two-step inference algorithm can more effectively correct model simulation biases and improve winter wheat yield estimation accuracy (R²=0.58, MAPE=12.75 %, and RMSE=1112 kg·ha⁻¹), outperforming the standard EnKF algorithm (R²=0.51, MAPE=14.62 %, RMSE=1328 kg·ha⁻¹). Overall, attributed to its unified approach to estimating both model parameters and state variables, the proposed two-step inference algorithm shows promising application prospects for data assimilation-based crop yield estimation.

Leveraging probability and statistical algorithms for enhanced financial risk management

This paper explores the foundations and applications of quantitative analysis in financial risk management. It examines the pivotal role of probability theory, statistical inference, and advanced algorithms in identifying, quantifying, and mitigating financial risks. Key concepts such as the Normal, Poisson, and Binomial distributions are discussed in the context of risk analysis, alongside statistical inference methods like hypothesis testing and confidence intervals. Furthermore, the paper investigates the application of portfolio optimization models, credit risk evaluation techniques, and market risk assessment methodologies in practical risk management scenarios. Additionally, it addresses the challenges posed by model risk, data quality, and regulatory compliance, emphasizing the need for rigorous validation, robust data governance, and ethical considerations in risk management practices. By integrating sophisticated quantitative techniques with real-world applications, financial institutions can enhance their ability to navigate the complexities of modern financial markets and achieve more effective risk management strategies.

YADA - Reference Free Deconvolution of RNA Sequencing Data

Introduction: We present YADA, a cellular content deconvolution algorithm for estimating cell type proportions in heterogeneous cell mixtures based on gene expression data. YADA utilizes curated gene signatures of cell type-specific marker genes, either obtained intrinsically from pure cell type expression matrices or provided by the user. Method: YADA implements an accessible and extensible deconvolution framework uniquely capable of handling marker genes alone as inputs. Adoption barriers are lowered significantly by relying solely on literature-supported cell type-specific signatures rather than full transcriptomic profiles from purified isolates. However, flexible inputs do not necessitate sacrificing rigor - predictions match metrics of current methodologies through an integrated optimization scheme balancing multiple inference algorithms. Efficiency optimizations via compiled runtimes enable rapid execution. Packaging as an importable Python toolkit promotes community enhancement while retaining codebase extensibility. Result: Validation studies demonstrate that YADA matches or exceeds the performance of current deconvolution methods on benchmark datasets. To demonstrate the utility and enable immediate usage, we provide an online Jupyter Notebook implementation coupled with tutorials. Conclusion: YADA provides an accurate, efficient, and extensible Python-based toolkit for cellular deconvolution analysis of heterogeneous gene expression data.

Complete inference via knowledge Petri nets and resolution rules

A novel inference approach is presented based on knowledge Petri nets and resolution rules. First, a knowledge base is modeled as a knowledge Petri net, where logical clauses are represented by monitor places, called semantic places. Second, a resolution pair is defined to identify a pair of semantic places, which, corresponding to two clauses that can be resolved, can be used to generate a new place in the knowledge Petri net. Such a new place represents an unknown clause implied by the knowledge base. Third, a method is proposed to decide whether a resolution inference is redundant based on the resolution pair. The size of a knowledge Petri net can be reduced, and the inference computation is saved in this way. Fourth, an inference algorithm is proposed employing the resolution pair and knowledge Petri net, proven to be sound and complete. Its computational complexity is polynomial with respect to the number of logical variables and clauses. In addition, another inference algorithm is developed for a given complete knowledge base and a set of newfound clauses. The algorithm is proven to be sound and complete, which offers significantly reduced computational complexity and specifically is linear concerning the number of new clauses. Finally, the wumpus world problem is taken as an example to illustrate and verify the proposed inference algorithms.

Genetic association between celiac disease and chronic kidney disease: a two-sample Mendelian randomization study

Objective A two-sample Mendelian randomization (MR) analysis was performed to elucidate the causal impact of celiac disease on the risk of chronic kidney disease (CKD). Methods The study comprised data from three genome-wide association studies involving individuals of European ancestry. The study groups included participants with celiac disease (n = 24,269), CKD (n = 117,165), and estimated glomerular filtration rate levels based on serum creatinine (eGFRcrea, n = 133,413). We employed four widely recognized causal inference algorithms: MR-Egger, inverse variance weighted (IVW), weighted median, and weighted mode. To address potential issues related to pleiotropy and overall effects, MR-Egger regression and the MR-PRESSO global test were performed. Heterogeneity was assessed using Cochran’s Q test. Results We identified 14 genetic variants with genome-wide significance. The MR analysis provided consistent evidence across the various methodologies, supporting a causal relationship between celiac disease and an elevated risk of CKD (odds ratio (OR)IVW = 1.027, p = 0.025; ORweighted median = 1.028, P = 0.049; ORweighted mode = 1.030, p = 0.044). Furthermore, we observed a causal link between celiac disease and a decreased eGFRcrea (ORIVW = 0.997, P = 2.94E-06; ORweighted median = 0.996, P = 1.68E-05; ORweighted mode = 0.996, P = 3.11E-04; ORMR Egger = 0.996, P = 5.00E-03). We found no significant evidence of horizontal pleiotropy, heterogeneity, or bias based on MR-Egger regression, MR-PRESSO, and Cochran’s Q test. Conclusion The results of this study indicate a causal relationship between celiac disease and an increased risk of CKD.

Probabilistic Mixture Model-Based Spectral Unmixing

Spectral unmixing attempts to decompose a spectral ensemble into the constituent pure spectral signatures (called endmembers) along with the proportion of each endmember. This is essential for techniques like hyperspectral imaging (HSI) used in environment monitoring, geological exploration, etc. Several spectral unmixing approaches have been proposed, many of which are connected to hyperspectral imaging. However, most extant approaches assume highly diverse collections of mixtures and extremely low-loss spectroscopic measurements. Additionally, current non-Bayesian frameworks do not incorporate the uncertainty inherent in unmixing. We propose a probabilistic inference algorithm that explicitly incorporates noise and uncertainty, enabling us to unmix endmembers in collections of mixtures with limited diversity. We use a Bayesian mixture model to jointly extract endmember spectra and mixing parameters while explicitly modeling observation noise and the resulting inference uncertainties. We obtain approximate distributions over endmember coordinates for each set of observed spectra while remaining robust to inference biases from the lack of pure observations and the presence of non-isotropic Gaussian noise. As a direct impact of our methodology, access to reliable uncertainties on the unmixing solutions would enable robust solutions to noise, as well as informed decision-making for HSI applications and other unmixing problems.

SEMIPARAMETRIC BIVARIATE HIERARCHICAL STATE SPACE MODEL WITH APPLICATION TO HORMONE CIRCADIAN RELATIONSHIP.

The adrenocorticotropic hormone and cortisol play critical roles in stress regulation and the sleep-wake cycle. Most research has been focused on how the two hormones regulate each other in terms of short-term pulses. Few studies have been conducted on the circadian relationship between the two hormones and how it differs between normal and abnormal groups. The circadian patterns are difficult to model as parametric functions. Directly extending univariate functional mixed effects models would result in a large dimensional problem and a challenging nonparametric inference. In this article, we propose a semi-parametric bivariate hierarchical state space model, in which each hormone profile is modeled by a hierarchical state space model, with nonparametric population-average and subject-specific components. The bivariate relationship is constructed by concatenating two latent independent subject-specific random functions specified by a design matrix, leading to a parametric inference on the correlation. We propose a computationally efficient state-space EM algorithm for estimation and inference. We apply the proposed method to a study of chronic fatigue syndrome and fibromyalgia and discover an erratic regulation pattern in the patient group in contrast to a circadian regulation pattern conforming to the day-night cycle in the control group.

BAYESIAN NESTED LATENT CLASS MODELS FOR CAUSE-OF-DEATH ASSIGNMENT USING VERBAL AUTOPSIES ACROSS MULTIPLE DOMAINS.

Understanding cause-specific mortality rates is crucial for monitoring population health and designing public health interventions. Worldwide, two-thirds of deaths do not have a cause assigned. Verbal autopsy (VA) is a well-established tool to collect information describing deaths outside of hospitals by conducting surveys to caregivers of a deceased person. It is routinely implemented in many low- and middle-income countries. Statistical algorithms to assign cause of death using VAs are typically vulnerable to the distribution shift between the data used to train the model and the target population. This presents a major challenge for analyzing VAs, as labeled data are usually unavailable in the target population. This article proposes a latent class model framework for VA data (LCVA) that jointly models VAs collected over multiple heterogeneous domains, assigns causes of death for out-of-domain observations and estimates cause-specific mortality fractions for a new domain. We introduce a parsimonious representation of the joint distribution of the collected symptoms using nested latent class models and develop a computationally efficient algorithm for posterior inference. We demonstrate that LCVA outperforms existing methods in predictive performance and scalability. Supplementary Material and reproducible analysis codes are available online. The R package LCVA implementing the method is available on GitHub (https://github.com/richardli/LCVA).

Opponent Exploitation Based on Bayesian Strategy Inference and Policy Tracking

In a multi-agent competitive environment, it is important for an agent to detect the opponent's policy and adopt a suitable policy to exploit the opponent. Conventionally, most methods, e.g., Bayesian Policy Reuse (BPR) variants, assume the opponent adopts a fixed policy or a randomly changing policy. In this paper, we make a more realistic and reasonable assumption that the opponent may select its policy based on the previous observation. Here, we define the term “strategy” as the mapping from the previous observation to the opponent's selected policy, and we propose the Bayesian Strategy Inference (BSI) framework to infer the opponent's strategy. Furthermore, to deal with opponents who may randomly select their policies, the BSI framework is combined with an intra-episode policy tracking mechanism to construct the Bayesian Strategy Inference plus Policy Tracking (BSI-PT) algorithm. In our experiments, we design an extended batter vs. pitcher game (EBvPG) for the evaluation of the proposed BSI-PT framework. The experimental results demonstrate that BSI-PT obtains higher policy prediction accuracy and winning percentage than three other BPR variants against the opponents with a specific policy selection strategy, with a random selection strategy, or with a partially random strategy.

Bayesian estimation of group event-related potential components (BEGEP): testing a model for synthetic and real datasets

Objective. The spatial resolution of event-related potentials (ERPs) recorded on the head surface is quite low, since the sensors located on the scalp register mixtures of signals from several cortical sources. Bayesian models for multi-channel ERPs obtained from a group of subjects under multiple task conditions can aid in recovering signals from these sources. Approach. This study introduces a novel model that captures several important characteristics of ERP, including person-to-person variability in the magnitude and latency of source signals. Furthermore, the model takes into account that ERP noise, the main source of which is the background electroencephalogram, has the following properties: it is spatially correlated, spatially heterogeneous, and varies over time and from person to person. Bayesian inference algorithms have been developed to estimate the parameters of this model, and their performance has been evaluated through extensive experiments using synthetic data and real ERPs records in a large number of subjects (N = 351). Main results. The signal estimates obtained using these algorithms were compared with the results of the analysis of ERPs by conventional methods. This comparison showed that the use of this model is suitable for the analysis of ERPs and helps to reveal some features of source signals that are difficult to observe in their mixture signals recorded on the scalp. Significance. This study shown that the proposed method is a potentially useful tool for analyzing ERPs collected from groups of subjects in various cognitive neuroscience experiments.

Visualization display and exit fault diagnosis of secondary virtual real circuit in intelligent substation

Most communication in smart substations uses fiber optic connections to connect secondary equipment now. Due to the lack of intuitive connection between devices, it leads to difficulties in monitoring and relay protection. In order to solve the problems of visualizing, monitoring, and diagnosing secondary virtual circuits in the operation and maintenance of intelligent substations, a comprehensive monitoring technology system for secondary virtual circuits in intelligent substations is constructed. Firstly, a breadth first search algorithm is proposed to sort out the physical connection relationships between secondary devices, which achieves visual display of secondary virtual circuits. Then, based on deep search fault inference algorithms, it can diagnose and locate the fault area. Through the massive data source of intelligent substations, multiple information fusion can be achieved. By applying the D-S evidence theory, precise fault localization can be achieved. Finally, the proof table method is used to determine the type of fault and to meet the practical application requirements for the operation and maintenance of secondary equipment in intelligent substations.

An information field theory approach to Bayesian state and parameter estimation in dynamical systems

Dynamical system state estimation and parameter calibration problems are ubiquitous across science and engineering. Bayesian approaches to the problem are the gold standard as they allow for the quantification of uncertainties and enable the seamless fusion of different experimental modalities. When the dynamics are discrete and stochastic, one may employ powerful techniques such as Kalman, particle, or variational filters. Practitioners commonly apply these methods to continuous-time, deterministic dynamical systems after discretizing the dynamics and introducing fictitious transition probabilities. However, approaches based on time-discretization suffer from the curse of dimensionality since the number of random variables grows linearly with the number of time-steps. Furthermore, the introduction of fictitious transition probabilities is an unsatisfactory solution because it increases the number of model parameters and may lead to inference bias. To address these drawbacks, the objective of this paper is to develop a scalable Bayesian approach to state and parameter estimation suitable for continuous-time, deterministic dynamical systems. Our methodology builds upon information field theory. Specifically, we parameterize the dynamical system state path using a linear basis. Then, we construct a physics-informed prior probability measure for the coefficients of the linear basis such that functions that satisfy the physics are more likely. This prior allows us to quantify model form errors. We connect the system's response to observations through a probabilistic model of the measurement process. The joint posterior over the system responses and all parameters is given by Bayes' rule. To approximate the intractable posterior, we develop a stochastic variational inference algorithm. In summary, the developed methodology offers a powerful framework for Bayesian estimation in dynamical systems.