Inference Algorithm Research Articles

MCMC for Bayesian nonparametric mixture modeling under differential privacy

Estimating the probability density of a population while preserving the privacy of individuals in that population is an important and challenging problem that has received considerable attention in recent years. While the previous literature focused on frequentist approaches, in this paper, we propose a Bayesian nonparametric mixture model under differential privacy (DP) and present two Markov chain Monte Carlo (MCMC) algorithms for posterior inference. One is a marginal approach, resembling Neal’s algorithm 5 with a pseudo-marginal Metropolis-Hastings move, and the other is a conditional approach. Although our focus is primarily on local DP, we show that our MCMC algorithms can be easily extended to deal with global differential privacy mechanisms. Moreover, for some carefully chosen mechanisms and mixture kernels, we show how auxiliary parameters can be analytically marginalized, allowing standard MCMC algorithms (i.e., non-privatized, such as Neal’s Algorithm 2) to be efficiently employed. Our approach is general and applicable to any mixture model and privacy mechanism. In several simulations and a real case study, we discuss the performance of our algorithms and evaluate different privacy mechanisms proposed in the frequentist literature.

Generating comprehensive lithium battery charging data with generative AI

In optimizing the performance and extending the lifespan of lithium batteries, accurate state prediction is crucial. Traditional regression and classification methods have achieved some success in predicting battery states. However, the effectiveness of these data-driven approaches largely depends on the availability and quality of public datasets. Additionally, generating electrochemical data primarily through battery experiments is a lengthy and costly process, making it extremely difficult to obtain high-quality data. This difficulty, combined with data incompleteness, significantly affects prediction accuracy. To address these challenges, this study introduces End of Life (EOL) and Equivalent Cycle Life (ECL) as supervised conditions for generative AI models. By integrating an embedding layer into the CVAE model, we developed the Refined Conditional Variational Autoencoder (RCVAE). Through preprocessing data into a quasi-video format, coupled with customized training and inference algorithms, the RCVAE generates the required charging data in real time based on the battery’s ECL and EOL. This avoids storing irrelevant information, saving space and resources. RCVAE uses a lightweight architecture, enabling fast generation of necessary voltage, current, temperature, and charging capacity data. This approach provides users with a comprehensive electrochemical dataset, pioneering a new research domain for the artificial synthesis of lithium battery data. Furthermore, based on the detailed synthetic data, various battery state indicators can be calculated, offering new perspectives and possibilities for lithium battery performance prediction.

Declarative probabilistic logic programming in discrete-continuous domains

Over the past three decades, the logic programming paradigm has been successfully expanded to support probabilistic modeling, inference and learning. The resulting paradigm of probabilistic logic programming (PLP) and its programming languages owes much of its success to a declarative semantics, the so-called distribution semantics. However, the distribution semantics is limited to discrete random variables only. While PLP has been extended in various ways for supporting hybrid, that is, mixed discrete and continuous random variables, we are still lacking a declarative semantics for hybrid PLP that not only generalizes the distribution semantics and the modeling language but also the standard inference algorithm that is based on knowledge compilation. We contribute the measure semantics together with the hybrid PLP language DC-ProbLog (where DC stands for distributional clauses) and its inference engine infinitesimal algebraic likelihood weighting (IALW). These have the original distribution semantics, standard PLP languages such as ProbLog, and standard inference engines for PLP based on knowledge compilation as special cases. Thus, we generalize the state of the art of PLP towards hybrid PLP in three different aspects: semantics, language and inference. Furthermore, IALW is the first inference algorithm for hybrid probabilistic programming based on knowledge compilation.

Revealing the coupled evolution process of construction risks in mega hydropower engineering through textual semantics

Mega hydropower engineering (MHE) involves complex construction processes, harsh environments, and highly mobile personnel, equipment, and other resources, resulting in a high incidence of construction risks and significant difficulties in safety control. Previous studies extensively explored the identification and governance of risks in MHE. However, these studies did not adequately utilize accident report case texts, and the coupled evolutionary law of construction risks remains unclear. To address this gap, this study developed a Bert-based Global Pointer Network for Named Entity-Relation Joint Extraction (Bert-GPLinker) to extract risk factors and their relationships. Second, the extracted risk factors and their relationships were employed to construct a knowledge graph, integrating text matching and path inference algorithms to propose a coupled evolutionary path reasoning method for construction risks. Finally, the coupled interactions among the risk factors were considered, and the interaction matrix principle was utilized to quantitatively assess the importance of each risk factor in the knowledge graph. The results showed that (1) the F1 value of Bert-GPLinker on the self-built dataset was 91.90%, indicating better model performance; (2) the coupled evolutionary reasoning system can quickly deduce the evolutionary path with the highest probability of any risk node and the types of accidents that may result; and (3) among the top 25 risk factors in terms of importance, those in the management system category accounted for 52%, with higher importance mainly concentrated in inspection, supervision, and training.

Self-aware collaborative edge inference with embedded devices for IIoT

Edge inference and other compute-intensive industrial Internet of Things (IIoT) applications suffer from a bad quality of experience due to the limited and heterogeneous computing and communication resources of embedded devices. To tackle these issues, we propose a model partitioning-based self-aware collaborative edge inference framework. Specifically, the device can adaptively adjust the local model inference scheme by sensing the available computing and communication resources of surrounding devices. When the inference latency requirement cannot be met by local computation, the model should be partitioned for collaborative computation on other devices to improve the inference efficiency. Furthermore, for two typical IIoT scenarios, i.e., bursting and stacking tasks, the latency-aware and throughput-aware collaborative inference algorithms are designed, respectively. Via jointly optimizing the partition layer and collaborative device selection, the optimal inference efficiency, characterized by minimum inference latency and maximum inference throughput, can be obtained. Finally, the performance of our proposal is validated through extensive simulations and tests conducted on 10 Raspberry Pi 4Bs using popular models. Specifically, in the case of two collaborative devices, our platform reaches up to 92.59% latency reduction for bursting tasks and 16.19× throughput growth for stacking tasks. In addition, the divergence between simulations and tests ranges from 1.64% to 9.56% for bursting tasks and from 3.24% to 11.24% for stacking tasks, which indicates that the theoretical performance analyses are solid. For the general case where the data privacy is not considered and the number of collaborative devices is optimally determined, up to 14.76× throughput speed up and 84.04% latency reduction can be obtained.

Partially typed multiparty sessions with internal delegation

A multiparty session formalises a set of concurrent communicating participants. The possibility for a participant to delegate some interactions to another participant is crucial for the expressivity of multiparty sessions. We propose the first type system for multiparty sessions with delegation where some communications between participants can be ignored. This allows us to type some sessions with global types representing interesting protocols, which have no type in the standard type systems. Our type system enjoys Subject Reduction, Session Fidelity and partial Lock-freedom. The last property ensures the absence of locks for participants with non-ignored communications. A sound and complete type inference algorithm is also discussed.

COFFEE: consensus single cell-type specific inference for gene regulatory networks.

The inference of gene regulatory networks (GRNs) is crucial to understanding the regulatory mechanisms that govern biological processes. GRNs may be represented as edges in a graph, and hence, it have been inferred computationally for scRNA-seq data. A wisdom of crowds approach to integrate edges from several GRNs to create one composite GRN has demonstrated improved performance when compared with individual algorithm implementations on bulk RNA-seq and microarray data. In an effort to extend this approach to scRNA-seq data, we present COFFEE (COnsensus single cell-type speciFic inFerence for gEnE regulatory networks), a Borda voting-based consensus algorithm that integrates information from 10 established GRN inference methods. We conclude that COFFEE has improved performance across synthetic, curated, and experimental datasets when compared with baseline methods. Additionally, we show that a modified version of COFFEE can be leveraged to improve performance on newer cell-type specific GRN inference methods. Overall, our results demonstrate that consensus-based methods with pertinent modifications continue to be valuable for GRN inference at the single cell level. While COFFEE is benchmarked on 10 algorithms, it is a flexible strategy that can incorporate any set of GRN inference algorithms according to user preference. A Python implementation of COFFEE may be found on GitHub: https://github.com/lodimk2/coffee.

Destination and time-series inference of moving objects via conditionally Markov process

This paper presents a destination and time-series inference algorithm for tracking moving targets. The destination of the object is considered the intent, and inference and state estimation are performed in the Bayesian framework. To describe the destination-aware target motion, we construct the state transition model using a conditionally Markov process. We introduce a multiple model to achieve simultaneous intent and time-series inferences. Given finite destination candidates, the maximum a posteriori hypothesis is chosen as the destination. For time-series inference, local estimates obtained from Kalman filters are fused to yield target state estimates. To address unspecified terminal conditions, the proposed algorithm incorporates parameter correction techniques based on relative geometry. Numerical simulations are performed to validate the proposed inference algorithm.

Using Early Rejection Markov Chain Monte Carlo and Gaussian Processes to Accelerate ABC Methods

Approximate Bayesian computation (ABC) is a class of Bayesian inference algorithms that targets problems with intractable or unavailable likelihood functions. It uses synthetic data drawn from the simulation model to approximate the posterior distribution. However, ABC is computationally intensive for complex models in which simulating synthetic data is very expensive. In this article, we propose an early rejection Markov chain Monte Carlo (ejMCMC) sampler based on Gaussian processes to accelerate inference speed. We early reject samples in the first stage of the kernel using a discrepancy model, in which the discrepancy between the simulated and observed data is modeled by Gaussian process (GP). Hence, synthetic data is generated only if the parameter space is worth exploring. We demonstrate through theory, simulation experiments, and real data analysis that the new algorithm significantly improves inference efficiency compared to existing early-rejection MCMC algorithms. In addition, we employ our proposed method within an ABC sequential Monte Carlo (SMC) sampler. In our numerical experiments, we use examples of ordinary differential equations, stochastic differential equations, and delay differential equations to demonstrate the effectiveness of the proposed algorithm. We develop an R package that is available at https://github.com/caofff/ejMCMC.

Model‐based clustering in simple hypergraphs through a stochastic blockmodel

AbstractWe propose a model to address the overlooked problem of node clustering in simple hypergraphs. Simple hypergraphs are suitable when a node may not appear multiple times in the same hyperedge, such as in co‐authorship datasets. Our model generalizes the stochastic blockmodel for graphs and assumes the existence of latent node groups and hyperedges are conditionally independent given these groups. We first establish the generic identifiability of the model parameters. We then develop a variational approximation Expectation‐Maximization algorithm for parameter inference and node clustering, and derive a statistical criterion for model selection. To illustrate the performance of our R package HyperSBM, we compare it with other node clustering methods using synthetic data generated from the model, as well as from a line clustering experiment and a co‐authorship dataset.

Jaxspec : A fast and robust Python library for X-ray spectral fitting

Inferring spectral parameters from X-ray data is one of the cornerstones of high-energy astrophysics, and is achieved using software stacks that have been developed over the last 20 years and more. However, as models get more complex and spectra are obtained with higher resolutions, these established software solutions become more feature-heavy, difficult to maintain and less efficient. We present jaxspec a Python package for performing this task quickly and robustly in a fully Bayesian framework. Based on the JAX ecosystem jaxspec allows the generation of differentiable likelihood functions compilable on core or graphical process units (GPUs), enabling the use of robust algorithms for Bayesian inference. We demonstrate the effectiveness of jaxspec samplers, in particular the no U-turn sampler, using a composite model and comparing what we obtain with the existing frameworks. We also demonstrate its ability to process high-resolution spectroscopy data using original methods by reproducing the results of the Hitomi collaboration on the Perseus cluster, while solving the inference problem using variational inference on a GPU. We obtain identical results when compared to other software and approaches, meaning that jaxspec provides reliable results while being $ 10$ times faster than existing alternatives. In addition, we show that variational inference can produce convincing results even on high-resolution data in less than 10 minutes on a GPU. With this package, we aim to pursue the goal of opening up X-ray spectroscopy to the existing ecosystem of machine learning and Bayesian inference, enabling researchers to apply new methods to solve increasingly complex problems in the best possible way. Our long-term ambition is the scientific exploitation of the data from the newAthena X-ray Integral Field Unit (X-IFU).

Transcriptome-Wide Root Causal Inference.

Root causal genes correspond to the first gene expression levels perturbed during pathogenesis by genetic or non-genetic factors. Targeting root causal genes has the potential to alleviate disease entirely by eliminating pathology near its onset. No existing algorithm discovers root causal genes from observational data alone. We therefore propose the Transcriptome-Wide Root Causal Inference (TWRCI) algorithm that identifies root causal genes and their causal graph using a combination of genetic variant and unperturbed bulk RNA sequencing data. TWRCI uses a novel competitive regression procedure to annotate cis and trans-genetic variants to the gene expression levels they directly cause. The algorithm simultaneously recovers a causal ordering of the expression levels to pinpoint the underlying causal graph and estimate root causal effects. TWRCI outperforms alternative approaches across a diverse group of metrics by directly targeting root causal genes while accounting for distal relations, linkage disequilibrium, patient heterogeneity and widespread pleiotropy. We demonstrate the algorithm by uncovering the root causal mechanisms of two complex diseases, which we confirm by replication using independent genome-wide summary statistics.

Bus travel feature inference with small samples based on multi-clustering topic model over Internet of Things

With the widespread application of Internet of Things (IoT) technology, there has been a shift from a broad-brush to a more refined approach in traffic optimization. An increasing amount of IoT data is being utilized in trajectory mining and inference, offering more precise characteristic information for optimizing public transportation. Services that optimize public transit based on inferred travel characteristics can enhance the appeal of public transport, increase its likelihood as a travel choice, alleviate traffic congestion, and reduce carbon emissions. However, the inherent complexities of disorganized and unstructured public transportation data pose significant challenges to extracting travel features. This study explores the enhancement of bus travel by integrating advanced technologies like positioning systems, IoT, and AI to infer features in public transportation data. It introduces the MK-LDA (MeanShift Kmeans Latent Dirichlet Allocation), a novel thematic modeling technique for deducing characteristics of public transit travel using limited travel trajectory data. The model employs a segmented inference methodology, initially leveraging the Mean-shift clustering algorithm to create POI seeds, followed by the P-K-means algorithm for discerning patterns in user travel behavior and extracting travel modalities. Additionally, a P-LDA (POI-Latent Dirichlet Allocation) inference algorithm is proposed to examine the interplay between travel characteristics and behaviors, specifically targeting attributes significantly correlated with public transit usage, including age, occupation, gender, activity levels, cost, safety, and personality traits. Empirical validation highlights the efficacy of this thematic modeling-based inference technique in identifying and predicting travel characteristics and patterns, boasting enhanced interpretability and outperforming conventional benchmarks.

Valid Inference After Causal Discovery

Causal discovery and causal effect estimation are two fundamental tasks in causal inference. While many methods have been developed for each task individually, statistical challenges arise when applying these methods jointly: estimating causal effects after running causal discovery algorithms on the same data leads to “double dipping,” invalidating the coverage guarantees of classical confidence intervals. To this end, we develop tools for valid post-causal-discovery inference. Across empirical studies, we show that a naive combination of causal discovery and subsequent inference algorithms leads to highly inflated miscoverage rates; on the other hand, applying our method provides reliable coverage while achieving more accurate causal discovery than data splitting.

Detection of Internal Hemorrhage via Sequential Inference: An In Silico Feasibility Study.

This paper investigates the feasibility of detecting and estimating the rate of internal hemorrhage based on continuous noninvasive hematocrit measurement. A unique challenge in hematocrit-based hemorrhage detection is that hematocrit decreases in response to hemorrhage and resuscitation with fluids, which makes hemorrhage detection during resuscitation challenging. We developed two sequential inference algorithms for detection of internal hemorrhage based on the Luenberger observer and the extended Kalman filter. The sequential inference algorithms use fluid resuscitation dose and hematocrit measurement as inputs to generate signatures to enable detection of internal hemorrhage. In the case of the extended Kalman filter, the signature is nothing but inferred hemorrhage rate, which allows it to also estimate internal hemorrhage rate. We evaluated the proof-of-concept of these algorithms based on in silico evaluation in 100 virtual patients subject to diverse hemorrhage and resuscitation rates. The results showed that the sequential inference algorithms outperformed naïve internal hemorrhage detection based on the decrease in hematocrit when hematocrit noise level was 1% (average F1 score: Luenberger observer 0.80; extended Kalman filter 0.76; hematocrit 0.59). Relative to the Luenberger observer, the extended Kalman filter demonstrated comparable internal hemorrhage detection performance and superior accuracy in estimating the hemorrhage rate. The analysis of the dependence of the sequential inference algorithms on measurement noise and plant parametric uncertainty showed that small (≤1%) hematocrit noise level and personalization of sequential inference algorithms may enable continuous noninvasive detection of internal hemorrhage and estimation of its rate.

Diagnosa Penyakit Kelamin (Vulvodynia) pada Wanita Menggunakan Metode Certainty Factor

Vulvodynia is a chronic pain condition affecting the vulva that significantly impacts women’s quality of life. Accurate and early diagnosis poses a challenge due to the often-overlapping symptoms with other conditions and the lack of definitive diagnostic tests. This paper proposes the use of expert system methods as a diagnostic tool for vulvodynia in women. The expert system, integrating medical knowledge with inference algorithms, is designed to analyze symptoms, medical history, and test results to provide accurate diagnoses and treatment recommendations. The study involves the development and evaluation of a computer-based expert system prototype that uses clinical data and medical decision-making to enhance the accuracy of vulvodynia diagnosis. Preliminary results indicate that the expert system can improve diagnostic rates and reduce the time required for identifying this condition, offering a potentially valuable tool for medical professionals in clinical practice.

Tensor product algorithms for inference of contact network from epidemiological data

We consider a problem of inferring contact network from nodal states observed during an epidemiological process. In a black-box Bayesian optimisation framework this problem reduces to a discrete likelihood optimisation over the set of possible networks. The cardinality of this set grows combinatorially with the number of network nodes, which makes this optimisation computationally challenging. For each network, its likelihood is the probability for the observed data to appear during the evolution of the epidemiological process on this network. This probability can be very small, particularly if the network is significantly different from the ground truth network, from which the observed data actually appear. A commonly used stochastic simulation algorithm struggles to recover rare events and hence to estimate small probabilities and likelihoods. In this paper we replace the stochastic simulation with solving the chemical master equation for the probabilities of all network states. Since this equation also suffers from the curse of dimensionality, we apply tensor train approximations to overcome it and enable fast and accurate computations. Numerical simulations demonstrate efficient black-box Bayesian inference of the network.

Source inference for misinformation spreading on hypergraphs

Source inference aims at revealing the seed of the misinformation spreading on social networks, and attracted great attention in the field of network science and cybersecurity. Extensive real-world data analyses have certificated that individual interactions exist pairwise and higher-order interactions, and thus should be described using the hypergraph. Previous studies about the source inference algorithms are mainly focused on simple graphs (i.e., a graph only has pairwise interactions) while neglecting the higher-order interactions. In this article, we propose a dynamical message-passing (DMP) algorithm to infer the misinformation spreading on hypergraphs. As a comparison, we also extend jordan centrality (JC), betweenness centrality (BC), closeness centrality (CC), and eigenvector centrality (EIG) algorithm to hypergraphs. The results show that our proposed DMP algorithm can accurately and effectively infer the propagation source on both artificial and real-world hypergraphs compared to the centrality algorithms. Even if the increase in propagation scale renders the other centrality algorithms completely ineffective, the DMP algorithm can still distinguish the real propagation sources in the form of a rank smaller than 10% of the network size. In general, our DMP algorithm provides an effective solution for inferring the misinformation source on high-order networks.

A Modular Service for Causal Inference in the Context of the Gaia-X Ecosystem.

Causal inference seeks to learn the effect of interventions on outcomes. Its potential in the health domain has been dramatically increasing recently, due to advancements in machine learning, as well as in the growing amount of medical data collected. Gaia-X provides a framework to implement Health Data Spaces at scale, in a compliant, secure, and trustable manner. In this paper, we provide a modular online service for causal inference using observational data, building on the Gaia-X framework. While two versions of the IDA algorithm for causal inference are already provided, the service allows users to further contribute algorithms in various programming languages (not only for causal inference), as well as their data, and efficiently execute these on a central server. Additionally, the platform facilitates the exchange of algorithms and data among participants. Users of the platform can enter into agreements with other users over the use of algorithms and data. Calculations can be carried out directly on the platform without the need to locally store foreign data.

Manifold Gaussian Variational Bayes on the Precision Matrix.

We propose an optimization algorithm for variational inference (VI) in complex models. Our approach relies on natural gradient updates where the variational space is a Riemann manifold. We develop an efficient algorithm for gaussian variational inference whose updates satisfy the positive definite constraint on the variational covariance matrix. Our manifold gaussian variational Bayes on the precision matrix (MGVBP) solution provides simple update rules, is straightforward to implement, and the use of the precision matrix parameterization has a significant computational advantage. Due to its black-box nature, MGVBP stands as a ready-to-use solution for VI in complex models. Over five data sets, we empirically validate our feasible approach on different statistical and econometric models, discussing its performance with respect to baseline methods.