Inference Problem Research Articles

Regularized Variational Approximation for Partially Confirmatory Factor Analysis

The recently developed partially confirmatory factor analysis (PCFA) framework offers a promising solution that effectively accommodates varying levels of available knowledge, blending the strengths of both data- and theory-driven approaches. Nevertheless, its reliance on Markov Chain Monte Carlo (MCMC) techniques for parameter estimation often comes with challenges due to its computationally intensive nature and issues related to convergence. To address that, this study introduces a novel and compelling alternative - the regularized variational approximation approach within the PCFA framework (PCFA-VA). The proposed method integrates regularization techniques and converts the classical Bayesian inference problem into an optimization problem by approximating the posterior distribution with a predefined family of distributions and employing Kullback–Leibler (KL) divergence to quantify such differences. In this sense, PCFA-VA avoids the complex integration problem required to obtain the exact posterior distribution and results in substantial computational savings. Based on the simulated and real data analysis, PCFA-VA has demonstrated the potential to achieve considerable accuracy while maintaining computational efficiency, making it scalable for large-scale problems.

AutoGFI: Streamlined Generalized Fiducial Inference for Modern Inference Problems in Models with Additive Errors

The concept of fiducial inference was introduced by R. A. Fisher in the 1930s to address the perceived limitations of Bayesian inference, particularly the need for subjective prior distributions in cases with limited prior information. However, Fisher’s fiducial approach lost favor due to complications, especially in multi-parameter problems. With renewed interest in fiducial inference in the 2000s, generalized fiducial inference (GFI) emerged as a promising extension of Fisher’s ideas, offering new solutions for complex inference challenges. Despite its potential, GFI’s adoption has been hindered by demanding mathematical derivations and complex implementation requirements, such as Markov Chain Monte Carlo (MCMC) algorithms. This paper introduces AutoGFI, a streamlined variant of GFI designed to simplify its application across various inference problems with additive noise. AutoGFI ’s accessibility lies in its simplicity—requiring only a fitting routine—making it a feasible option for a wider range of researchers and practitioners. To demonstrate its efficacy, AutoGFI is applied to three challenging problems: tensor regression, matrix completion, and network cohesion regression. These case studies showcase AutoGFI ’s competitive performance against specialized solutions, highlighting its potential to broaden the application of GFI in practical domains, ultimately enriching the statistical inference toolkit.

Hierarchical Bayesian Data Selection

There are many issues that can cause problems when attempting to infer model parameters from data. Data and models are both imperfect, and as such there are multiple scenarios in which standard methods of inference will lead to misleading conclusions; corrupted data, models which are only representative of subsets of the data, or multiple regions in which the model is best fit using different parameters. Methods exist for the exclusion of some anomalous types of data, but in practice, data cleaning is often undertaken by hand before attempting to fit models to data. In this work, we will employ hierarchical Bayesian data selection; the simultaneous inference of both model parameters, and parameters which represent our belief that each observation within the data should be included in the inference. The aim, within a Bayesian setting, is to find the regions of observation space for which the model can well-represent the data, and to find the corresponding model parameters for those regions. A number of approaches will be explored, and applied to test problems in linear regression, and to the problem of fitting an ODE model, approximated by a finite difference method. The approaches are simple to implement, can aid mixing of Markov chains, and are broadly applicable to many inferential problems.

A statistical modelling approach for determining the cause of reported respiratory syndromes from internet-based participatory surveillance when influenza virus and SARS-CoV-2 are co-circulating.

Symptom-only case definitions are insufficient to discriminate COVID-like illness from acute respiratory infection (ARI) or influenza-like illness (ILI), due to the overlap in case definitions. Our objective was to develop a statistical method that does not rely on case definitions to determine the contribution of influenza virus and SARS-CoV-2 to the ARI burden during periods when both viruses are circulating. Data sources used for testing the approach were weekly ARI syndrome reports from the Infectieradar participatory syndromic surveillance system during the analysis period (the first 25 weeks of 2022, in which SARS-CoV-2 and influenza virus co-circulated in the Netherlands) and data from virologically tested ARI (including ILI) patients who consulted a general practitioner in the same period. Estimation of the proportions of ARI attributable to influenza virus, SARS-CoV-2, or another cause was framed as an inference problem, through which all data sources are combined within a Bayesian framework to infer the weekly numbers of ARI reports attributable to each cause. Posterior distributions for the attribution proportions were obtained using Markov Chain Monte-Carlo methods. Application of the approach to the example data sources indicated that, of the total ARI reports (total of 11,312; weekly mean of 452) during the analysis period, the model attributed 35.4% (95% CrI: 29.2-40.0%) and 27.0% (95% CrI: 19.3-35.2%) to influenza virus and SARS-CoV-2, respectively. The proposed statistical model allows the attribution of respiratory syndrome reports from participatory surveillance to either influenza virus or SARS-CoV-2 infection in periods when both viruses are circulating, but comparability of the participatory surveillance and virologically tested populations is important. Portability for use by other countries with established participatory respiratory surveillance systems is an asset.

Inferences of accelerated generalized type-I hybrid censoring data under power hazard rate population

In reliability engineering or medical studies, more information about the life expectancy of products or materials is needed. This information under use conditions is more difficult to collect especially under highly reliable case. In this paper, the problem of statistical inference when the lifetime of a product or material has the power failure rate distribution is developed. To save the cost and time induced by units the experiment is designed with respect to partially step-stress accelerated life tests (ALTs) with type-I generalized hybrid censoring scheme (GHCS). The point estimate of the model parameters as well as the accelerated factor are obtained with respect to the maximum likelihood (ML) and Bayes methods. Also, the asymptotic normal properties of ML estimates, two bootstrap techniques and Markov Chain Monte Carlo techniques are used to formulate interval estimators. A real lifetime data set is used to illustrate the application of the proposed model. Finally, a Monte Carlo simulation study is constructed to discuss and assess, the proposed model and estimation methods.

Inference methods for multivariate coefficient of variation: a novel NPC-based approach

In many fields, from psychology and biology to quality control, logistics and risk management, the coefficient of variation (CV) and in particular the multivariate coefficient of variation (MCV), can be applied as measures of variability. Consequently, several parametric and nonparametric inference methods for CV and MCV have been introduced in the literature. These include a recently introduced permutation-based James-type studentized statistic for making inferences about MCV. In this article, we propose a new permutation-based solution exploiting the Non Parametric Combination (NPC) methodology. This new extension presents some major advantages: it can be applied to test both the one and two-sided ordered alternatives, it can deal with high-dimensional settings, even in the case of small sample sizes, and it can be used to deal with isotonic inference problems. In this paper, we present an extensive simulation study to address both one-way Multivariate Analysis of Variance (MANOVA)-like problems and isotonic inference problems. A case study is then provided to show a real data application of the new NPC-based solution.

Top-down proteomics of venoms from five Micrurus species from Costa Rica: Comparative composition of phospholipase A2-rich vs three-finger toxin-rich phenotypes

Coralsnakes of the genus Micrurus include more than 80 species distributed in the American continent. They produce potent neurotoxic venoms acting at the neuromuscular junction and potentially leading to respiratory paralysis and death. The vast majority of proteins in coralsnake venoms belong to the three-finger toxin (3FTx) and the group I phospholipase A2 (PLA2) families. Previous studies using ‘bottom-up’ proteomic strategies have revealed a compositional dichotomy of toxin expression by which different Micrurus species display a predominance of either 3FTx or PLA2 proteins in their venoms, possibly linked to the phylogeographic structure of the genus radiation. ‘Top-down’ proteomics (TDP) allows the direct analysis of intact proteins in a high resolution mass spectrometer, circumventing the limitations of the ‘peptide-to-protein inference problem’ inherent to the bottom-up approach. Here, we analyzed the venoms of five out of the six Micrurus species that inhabit Costa Rica, by using a TDP approach. Results unveil venom proteoforms that are shared between these species, and provide additional insights into the variable compositional complexity of these venoms and relationships to their 3FTx/PLA2 dichotomy.

Bayesian Mechanics of Synaptic Learning Under the Free-Energy Principle.

The brain is a biological system comprising nerve cells and orchestrates its embodied agent's perception, behavior, and learning in dynamic environments. The free-energy principle (FEP) advocated by Karl Friston explicates the local, recurrent, and self-supervised cognitive dynamics of the brain's higher-order functions. In this study, we continue to refine the FEP through a physics-guided formulation; specifically, we apply our theory to synaptic learning by considering it an inference problem under the FEP and derive the governing equations, called Bayesian mechanics. Our study uncovers how the brain infers weight changes and postsynaptic activity, conditioned on the presynaptic input, by deploying generative models of the likelihood and prior belief. Consequently, we exemplify the synaptic efficacy in the brain with a simple model; in particular, we illustrate that the brain organizes an optimal trajectory in neural phase space during synaptic learning in continuous time, which variationally minimizes synaptic surprisal.

Parameter estimation for allometric trophic network models: A variational Bayesian inverse problem approach

Abstract Differential equation models are powerful tools for predicting biological systems, capable of projecting far into the future and incorporating data recorded at arbitrary times. However, estimating these models' parameters from observations can be challenging because numerical methods are often required to approximate their solution. An example of such a model is the allometric trophic network model, for which studies considering its inverse problem are limited, particularly in the Bayesian framework. Here we develop a variational Bayesian method for parameter inference of the allometric trophic network model and explore how accurately we can recover its parameter values. We represent the model as a Bayesian neural network, which combines an artificial neural network with Bayesian inference, using a surrogate for the posterior distribution of model parameters, and train this model by evolutionary optimization to avoid potentially costly computation of the gradient with respect to the model parameters. Using synthetic data, we compare the accuracy of this variational inference to ordinary least squares estimation. To reduce the number of estimated parameters, we focus on the inference of functional response parameters. Our variational Bayesian method yields parameter estimates that are comparable to the ordinary least squares results in terms of accuracy. The method provides a promising approach for including uncertainty quantification in parameter estimation, which the simple ordinary least squares approach as it is does not address. Regardless of the method, potential multimodality of the inference problem is nonetheless important to keep in mind. The present study suggests a technique for parameter inference of ordinary differential equation models in the Bayesian context. We propose the method especially for validation of the allometric trophic network model against empirical data.

A generic random field approach for stratification uncertainty quantification

Geological uncertainty is a crucial factor in the reliability design of geotechnical structures, necessitating advanced probabilistic approaches for robust subsurface stratification and uncertainty quantification. However, the spatial existence of geological formations at a specific location is indeed a nominal categorical variable. Unlike the spatial variability of soil properties that has been widely addressed using random field theory or geostatistics, quantifying the spatial correlations of formation existence poses unique challenges. In this study, a generic framework for geological uncertainty quantification is proposed. The spatial existence of each formation is modeled with individual latent random fields, such that the spatial correlation strength of formation existence can be defined in a consistent manner as that in geotechnical random fields. Moreover, an effective distance metric is introduced to accommodate spatial correlations within a distorted random field, allowing for the incorporation of stratigraphic dip naturally. The latent random fields are subsequently input into the generalized linear mixed model (GLMM) to derive the probabilistic distribution of formation existence in the form of discrete random fields. The simulation framework is further formulated as a Bayesian inference problem to incorporate existing information from borehole records of a site. The proposed formulation offers great flexibility and versatility in simulating various complex stratigraphic configurations, which is demonstrated through a hypothetical case study with an inter-layer and a real case study in Perth, West Australia. This study advances rigorous quantification of stratification uncertainty, facilitating more reliable geotechnical design and risk assessment in complex geological environments.

Simulation-based, Finite-sample Inference for Privatized Data

Privacy protection methods, such as differentially private mechanisms, introduce noise into resulting statistics which often produces complex and intractable sampling distributions. In this paper, we propose a simulation-based “repro sample” approach to produce statistically valid confidence intervals and hypothesis tests, which builds on the work of Xie and Wang (2022). We show that this methodology is applicable to a wide variety of private inference problems, appropriately accounts for biases introduced by privacy mechanisms (such as by clamping), and improves over other state-of-the-art inference methods such as the parametric bootstrap in terms of the coverage and type I error of the private inference. We also develop significant improvements and extensions for the repro sample methodology for general models (not necessarily related to privacy), including 1) modifying the procedure to ensure guaranteed coverage and type I errors, even accounting for Monte Carlo error, and 2) proposing efficient numerical algorithms to implement the confidence intervals and p-values.

Heuristic satisficing inferential decision making in human and robot active perception.

Inferential decision-making algorithms typically assume that an underlying probabilistic model of decision alternatives and outcomes may be learned a priori or online. Furthermore, when applied to robots in real-world settings they often perform unsatisfactorily or fail to accomplish the necessary tasks because this assumption is violated and/or because they experience unanticipated external pressures and constraints. Cognitive studies presented in this and other papers show that humans cope with complex and unknown settings by modulating between near-optimal and satisficing solutions, including heuristics, by leveraging information value of available environmental cues that are possibly redundant. Using the benchmark inferential decision problem known as "treasure hunt", this paper develops a general approach for investigating and modeling active perception solutions under pressure. By simulating treasure hunt problems in virtual worlds, our approach learns generalizable strategies from high performers that, when applied to robots, allow them to modulate between optimal and heuristic solutions on the basis of external pressures and probabilistic models, if and when available. The result is a suite of active perception algorithms for camera-equipped robots that outperform treasure-hunt solutions obtained via cell decomposition, information roadmap, and information potential algorithms, in both high-fidelity numerical simulations and physical experiments. The effectiveness of the new active perception strategies is demonstrated under a broad range of unanticipated conditions that cause existing algorithms to fail to complete the search for treasures, such as unmodelled time constraints, resource constraints, and adverse weather (fog).

Terraces in species tree inference from gene trees

A terrace in a phylogenetic tree space is a region where all trees contain the same set of subtrees, due to certain patterns of missing data among the taxa sampled, resulting in an identical optimality score for a given data set. This was first investigated in the context of phylogenetic tree estimation from sequence alignments using maximum likelihood (ML) and maximum parsimony (MP). It was later extended to the species tree inference problem from a collection of gene trees, where a set of equally optimal species trees was referred to as a “pseudo” species tree terrace which does not consider the topological proximity of the trees in terms of the induced subtrees resulting from certain patterns of missing data. In this study, we mathematically characterize species tree terraces and investigate the mathematical properties and conditions that lead multiple species trees to induce/display an identical set of locus-specific subtrees owing to missing data. We report that species tree terraces are agnostic to gene tree heterogeneity. Therefore, we introduce and characterize a special type of gene tree topology-aware terrace which we call “peak terrace”. Moreover, we empirically investigated various challenges and opportunities related to species tree terraces through extensive empirical studies using simulated and real biological data. We demonstrate the prevalence of species tree terraces and the resulting ambiguity created for tree search algorithms. Remarkably, our findings indicate that the identification of terraces could potentially lead to advances that enhance the accuracy of summary methods and provide reasonably accurate branch support.

Strict monotonicity of stochastic process extreme distributions

Strict monotonicity is proved for the distributions of extremes of processes consisting of series of bounded function with independent random coefficients, in particular for zero mean continuous Gaussian processes over compact metric space. These results have wide applications to global inference problems on unknown functions.

Alteration detection of tensor dependence structure via sparsity-exploited reranking algorithm

Tensor-valued data arise frequently from a wide variety of scientific applications, and many among them can be translated into an alteration detection problem of tensor dependence structures. In this article, we formulate the problem under the popularly adopted tensor-normal distributions and aim at two-sample correlation/partial correlation comparisons of tensor-valued observations. Through decorrelation and centralization, a separable covariance structure is employed to pool sample information from different tensor modes to enhance the power of the test. Additionally, we propose a novel Sparsity-Exploited Reranking Algorithm (SERA) to further improve the multiple testing efficiency. Such efficiency gain is achieved by incorporating a carefully constructed auxiliary tensor sequence to rerank the p-values. Besides the tensor framework, SERA is also generally applicable to a wide range of two-sample large-scale inference problems with sparsity structures, and is of independent interest. The asymptotic properties of the proposed test are derived and the algorithm is shown to control the false discovery at the pre-specified level. We demonstrate the efficacy of the proposed method through intensive simulations and two scientific applications.

Towards a Field-Based Bayesian Evidence Inference from Nested Sampling Data.

Nested sampling (NS) is a stochastic method for computing the log-evidence of a Bayesian problem. It relies on stochastic estimates of prior volumes enclosed by likelihood contours, which limits the accuracy of the log-evidence calculation. We propose to transform the prior volume estimation into a Bayesian inference problem, which allows us to incorporate a smoothness assumption for likelihood-prior-volume relations. As a result, we aim to increase the accuracy of the volume estimates and thus improve the overall log-evidence calculation using NS. The method presented works as a post-processing step for NS and provides posterior samples of the likelihood-prior-volume relation, from which the log-evidence can be calculated. We demonstrate an implementation of the algorithm and compare its results with plain NS on two synthetic datasets for which the underlying evidence is known. We find a significant improvement in accuracy for runs with less than one hundred active samples in NS but a proneness for numerical problems beyond this point.

Sparse representations, inference and learning

Abstract In recent years statistical physics has proven to be a valuable tool to probe into large dimensional inference problems such as the ones occurring in machine learning. Statistical physics provides analytical tools to study fundamental limitations in their solutions and proposes algorithms to solve individual instances. In these notes, based on the lectures by Marc Mézard in 2022 at the summer school in Les Houches, we will present a general framework that can be used in a large variety of problems with weak long-range interactions, including the compressed sensing problem, or the problem of learning in a perceptron. We shall see how these problems can be studied at the replica symmetric level, using developments of the cavity methods, both as a theoretical tool and as an algorithm.

Efficient approximate Bayesian inference for quantifying uncertainty in multiscale animal movement models

It is becoming increasingly important for wildlife managers and conservation ecologists to understand which resources are selected or avoided by an animal and how to best predict future spatial distributions of animal populations in the long term. However, inferring the patterns of space use by animals is a challenging multiscale inference problem, and formal uncertainty quantification of parameter estimates is an essential component of models that provide useful predictions across scales. In this study, we develop an approximate Bayesian inference framework for step selection models of animal movement which quantifies the uncertainty in estimates of resource selection and avoidance parameters within the Bayesian paradigm. The framework allows joint inference of movement and resource selection parameters of animals and is multiscale in that parameters inferred from fine scale movement steps scale to produce predictions of long-term patterns of space use. Our analysis focuses on simulated movement data in which we test the performance of our framework by altering movement parameters in the data-generating process. In our simulations, individuals respond to two environmental covariates and we employ all combinations of positive and negative selection coefficients corresponding to attraction to an environmental feature and avoidance of an environmental feature, respectively. In all scenarios, we recover the movement parameters used for the simulation of synthetic movement data using variational inference, an approximate Bayesian method, allowing us to formally quantify the uncertainty associated with each parameter for varying data set sizes. Our framework successfully recovered all combinations of movement parameters of the simulated data and accurately captured their posterior distributions given the available data suggesting that the framework is reliable and suitable for inferring how animals select resources and move on a landscape.Notably, our analysis shows that even for reasonably large data sets (circa 10,000 observations) there can still be considerable uncertainty associated with resource selection parameters which can in turn lead to inaccurate predictions of long term space use if not properly incorporated into the modelling approach. To further illustrate the utility of our approach, we also present a case study of its application to an example data set consisting of GPS locations of a fisher (Martes pennanti). Our approach will be of interest to ecologists looking to address conservation questions such as when and where animals are likely to spend most of their time. Furthermore, the approach could be used to predict new suitable areas for conservation based on how GPS collared animals use or avoid resources while including uncertainty around the predictions, thereby helping to make informed management decisions.

Overcoming Catastrophic Forgetting in Continual Fine-Grained Urban Flow Inference

Citywide fine-grained urban flow inference (FUFI) problem aims to infer the high-resolution flow maps from the coarse-grained ones, which plays an important role in sustainable and economic urban computing and intelligent traffic management. Previous models tackle this problem from spatial constraint, external factors, and memory cost. However, utilizing the new urban flow maps to calibrate the learned model is very challenging due to the “catastrophic forgetting” problem and is still under-explored. In this article, we make the first step in FUFI and present CUFAR—Continual Urban Flow inference with augmented Adaptive knowledge Replay—a novel framework for inferring the fine-grained citywide traffic flows. Specifically, (1) we design a spatial-temporal inference network that can extract better flow map features from both local and global levels; (2) then, we present an augmented adaptive knowledge replay (AKR) training algorithm to selectively replay the learned knowledge to facilitate the learning process of the model on new knowledge without forgetting. We apply several data augmentation techniques to improve the generalization capability of the learning model, gaining additional performance improvements. We also propose a knowledge discriminator to avoid the “negative replaying” issue introduced by noisy urban flow maps. Extensive experiments on two large-scale real-world FUFI datasets demonstrate that our proposed model consistently outperforms strong baselines and effectively mitigates the forgetting problem.

Awareness of DepEd Core Values among Grade 12 Students in a Public High School in the Philippines

This study determined the level of awareness of Department of Education (DepEd) core values among grade 12 students in a public school in Antique, Philippines, in the areas of maka-Diyos (Godly), makatao (person-oriented), makakalikasan (pro-environment), and makabansa (patriotism). Likewise, the study assessed the significant difference in their awareness of DepEd core values relative to their demographics. Hence, the study utilized quantitative research design, particularly descriptive-comparative approaches. The theoretical framework of the study is the ecological systems theory, which assumes the influence of a series of nested systems and environments in shaping human development. This indicates the need for considering the larger ecological context that drives core value awareness. Meanwhile, using the adopted questionnaire, data were collected from 293 Grade 12 senior high school students in a public high school in Antique, Philippines, during the school year 2023-2024. In the analysis of data, frequency, percentage, mean, standard deviation, Mann-Whitney U test, and Kruskal-Wallis test were employed to analyze both the descriptive and inferential problems. The study revealed that the overall core value awareness of students was very high (M=4.35, SD=0.48). This shows that the students at this public school fully recognized the four DepEd core values, namely, maka-Diyos, makatao, makakalikasan, and makabansa. Also, among all the four demographics, only sex [U=7782.5, P=0.001] and strand [χ2(5) =13.739, P=0.017] were found with significant differences. The findings postulate that to elicit students’ core value awareness, strengthening educational policies on consistent core value integration in school curricula, programs, and activities is greatly encouraged. The findings also convey the need to strengthen sustainable teacher orientation to maintain their students' high awareness.