Inference Problem Research Articles

Maximum Geometric Quantum Entropy.

Any given density matrix can be represented as an infinite number of ensembles of pure states. This leads to the natural question of how to uniquely select one out of the many, apparently equally-suitable, possibilities. Following Jaynes' information-theoretic perspective, this can be framed as an inference problem. We propose the Maximum Geometric Quantum Entropy Principle to exploit the notions of Quantum Information Dimension and Geometric Quantum Entropy. These allow us to quantify the entropy of fully arbitrary ensembles and select the one that maximizes it. After formulating the principle mathematically, we give the analytical solution to the maximization problem in a number of cases and discuss the physical mechanism behind the emergence of such maximum entropy ensembles.

Causal inference for psychologists who think that causal inference is not for them

AbstractCorrelation does not imply causation and psychologists' causal inference training often focuses on the conclusion that therefore experiments are needed—without much consideration for the causal inference frameworks used elsewhere. This leaves researchers ill‐equipped to solve inferential problems that they encounter in their work, leading to mistaken conclusions and incoherent statistical analyses. For a more systematic approach to causal inference, this article provides brief introductions to the potential outcomes framework—the “lingua franca” of causal inference—and to directed acyclic graphs, a graphical notation that makes it easier to systematically reason about complex causal situations. I then discuss two issues that may be of interest to researchers in social and personality psychology who think that formalized causal inference is of little relevance to their work. First, posttreatment bias: In various common scenarios (noncompliance, mediation analysis, missing data), researchers may analyze data from experimental studies in a manner that results in internally invalid conclusions, despite randomization. Second, tests of incremental validity: Routine practices in personality psychology suggest that they may be conducted for at least two different reasons (to demonstrate the non‐redundancy of new scales, to support causal conclusions) without being particularly suited for either purpose. Taking causal inference seriously is challenging; it reveals assumptions that may make many uncomfortable. However, ultimately it is a necessary step to ensure the validity of psychological research.

Probabilistic Brain Extraction in MR Images via Conditional Generative Adversarial Networks.

Brain extraction, or the task of segmenting the brain in MR images, forms an essential step for many neuroimaging applications. These include quantifying brain tissue volumes, monitoring neurological diseases, and estimating brain atrophy. Several algorithms have been proposed for brain extraction, including image-to-image deep learning methods that have demonstrated significant gains in accuracy. However, none of them account for the inherent uncertainty in brain extraction. Motivated by this, we propose a novel, probabilistic deep learning algorithm for brain extraction that recasts this task as a Bayesian inference problem and utilizes a conditional generative adversarial network (cGAN) to solve it. The input to the cGAN's generator is an MR image of the head, and the output is a collection of likely brain images drawn from a probability density conditioned on the input. These images are used to generate a pixel-wise mean image, serving as the estimate for the extracted brain, and a standard deviation image, which quantifies the uncertainty in the prediction. We test our algorithm on head MR images from five datasets: NFBS, CC359, LPBA, IBSR, and their combination. Our datasets are heterogeneous regarding multiple factors, including subjects (with and without symptoms), magnetic field strengths, and manufacturers. Our experiments demonstrate that the proposed approach is more accurate and robust than a widely used brain extraction tool and at least as accurate as the other deep learning methods. They also highlight the utility of quantifying uncertainty in downstream applications. Additional information and codes for our method are available at: https://github.com/bmri/bmri.

Novel symmetry-preserving neural network model for phylogenetic inference.

Scientists world-wide are putting together massive efforts to understand how the biodiversity that we see on Earth evolved from single-cell organisms at the origin of life and this diversification process is represented through the Tree of Life. Low sampling rates and high heterogeneity in the rate of evolution across sites and lineages produce a phenomenon denoted "long branch attraction" (LBA) in which long nonsister lineages are estimated to be sisters regardless of their true evolutionary relationship. LBA has been a pervasive problem in phylogenetic inference affecting different types of methodologies from distance-based to likelihood-based. Here, we present a novel neural network model that outperforms standard phylogenetic methods and other neural network implementations under LBA settings. Furthermore, unlike existing neural network models in phylogenetics, our model naturally accounts for the tree isomorphisms via permutation invariant functions which ultimately result in lower memory and allows the seamless extension to larger trees. We implement our novel theory on an open-source publicly available GitHub repository: https://github.com/crsl4/nn-phylogenetics.

Causal Agnosticism About Race: Variable Selection Problems in Causal Inference

Abstract This paper proposes a novel view in the the philosophy of race & causation literature known as “causal agnosticism” about race. Causal agnosticism about race implies that it is reasonable to refrain from making judgments about whether race is a cause. The paper’s thesis asserts that certain conditions must be met to infer that something is a cause, according to the fundamental assumptions of causal inference. However, in the case of race, these conditions are often violated. By advocating for causal agnosticism, the paper suggests a more modest approach to understanding the role of race in causal relationships.

Uncertainty quantification in multiaxial fatigue life prediction using Bayesian neural networks

In practical engineering scenarios, unforeseen failures may result in significant costs and risks. Assessing the uncertainty related to multiaxial fatigue life prediction is crucial for engineering applications. However, establishing a reliable multiaxial fatigue life prediction model remains challenging due to uncertainties in physical models, material properties, and measurement data. This paper proposes an effective Bayesian Neural Network (BNN) model for quantifying the uncertainty in multiaxial fatigue life prediction. Fatigue parameters in the multiaxial fatigue life prediction model are used as inputs for the BNN. The BNN method utilizes No-U-turn Sampler (NUTS) and Automatic Differentiation Variational Inference (ADVI) to address the inference problem in the model and perform life predictions for unknown non-proportional loading paths and different materials. Validation using experimental data from six different materials confirms the effectiveness of the BNN model. Experiments show that the performance of BNN models is beneficial in most cases compared to classical ML models, assessed through different error standards, statistical tests, Taylor diagrams, uncertainty analysis, and scatter index analysis. The proposed method provides a clear understanding of the uncertainty in multiaxial fatigue life prediction, offering a promising approach for simulating fatigue life under multiaxial loading in engineering applications.

Comparison of predictors under constrained general linear model and its future observations

. This study deals with some basic inference problems about future observations in a general linear model (GLM) with linear parameter constraints, known as a constrained general linear model (CGLM). Combining the CGLM and its future observations, the author turns the model into a reparameterized form. Using some quadratic matrix optimization methods, the author derives analytical formulas for calculating the best linear unbiased predictors (BLUPs) of all unknown parameter matrices under a CGLM and its future observations. In particular, the author next gives a comprehensive search on the comparison of dispersion matrices of BLUPs of unknown vectors by establishing various equalities and inequalities for dispersion matrices of BLUPs under the model by using elementary block matrix operations and some formulas of rank and inertia of block matrices.

The decimation scheme for symmetric matrix factorization

Matrix factorization is an inference problem that has acquired importance due to its vast range of applications that go from dictionary learning to recommendation systems and machine learning with deep networks. The study of its fundamental statistical limits represents a true challenge, and despite a decade-long history of efforts in the community, there is still no closed formula able to describe its optimal performances in the case where the rank of the matrix scales linearly with its size. In the present paper, we study this extensive rank problem, extending the alternative ‘decimation’ procedure that was recently introduced by Camilli and Mézard, and carry out a thorough study of its performance. Decimation aims at recovering one column/line of the factors at a time, by mapping the problem into a sequence of neural network models of associative memory at a tunable temperature. The main advantage of decimation is that its theoretical performance can be studied using statistical physics techniques. In this paper we provide the general analysis applying to a large class of compactly supported priors and we show that the replica symmetric free entropy of the neural network models takes a universal form in the low temperature limit. We then study the decimation phase diagrams in two concrete cases, the sparse Ising prior and the uniform prior, for which we show that matrix factorization is theoretically possible when the ration P/N of rank to variable is below a certain threshold. This suggest that a possible route to solving algorithmically the matrix factorization problem could be to find efficient decimation algorithms; in this respect, we show that simple simulated annealing, though being effective for limited signal sizes, is not scalable.

Bayesian model calibration for block copolymer self-assembly: Likelihood-free inference and expected information gain computation via measure transport

We consider the Bayesian calibration of models describing the phenomenon of block copolymer (BCP) self-assembly, which is to infer model parameters and their uncertainty from noisy image data produced by microscopy or X-ray scattering characterization of BCP structures. To account for the long-range disorder in BCP structures, we introduce auxiliary variables in the model to represent this aleatory uncertainty. These variables, however, result in an integrated likelihood function for high-dimensional image data that is generally intractable to evaluate. We tackle this Bayesian inference problem using a likelihood-free inference (LFI) approach based on measure transport together with the construction of summary statistics for the image data. We show that expected information gains (EIG) from observed data about the model parameters can be computed with no significant additional cost. Lastly, we present a numerical case study using the Ohta–Kawasaki model for diblock copolymer thin film self-assembly and top-down microscopy characterization. We introduce several domain-specific energy and Fourier-based summary statistics for calibration and quantify their informativeness using EIG. The effect of various data corruptions, summary statistics, and experimental designs on the calibration results are studied using the proposed LFI method.

Inferring networks from time series: A neural approach.

Network structures underlie the dynamics of many complex phenomena, from gene regulation and foodwebs to power grids and social media. Yet, as they often cannot be observed directly, their connectivities must be inferred from observations of the dynamics to which they give rise. In this work, we present a powerful computational method to infer large network adjacency matrices from time series data using a neural network, in order to provide uncertainty quantification on the prediction in a manner that reflects both the degree to which the inference problem is underdetermined as well as the noise on the data. This is a feature that other approaches have hitherto been lacking. We demonstrate our method's capabilities by inferring line failure locations in the British power grid from its response to a power cut, providing probability densities on each edge and allowing the use of hypothesis testing to make meaningful probabilistic statements about the location of the cut. Our method is significantly more accurate than both Markov-chain Monte Carlo sampling and least squares regression on noisy data and when the problem is underdetermined, while naturally extending to the case of nonlinear dynamics, which we demonstrate by learning an entire cost matrix for a nonlinear model of economic activity in Greater London. Not having been specifically engineered for network inference, this method in fact represents a general parameter estimation scheme that is applicable to any high-dimensional parameter space.

Machine learning in healthcare and the methodological priority of epistemology over ethics

ABSTRACT This paper develops an account of how the implementation of ML models into healthcare settings requires revising the methodological apparatus of philosophical bioethics. On this account, ML models are cognitive interventions that provide decision-support to physicians and patients. Due to reliability issues, opaque reasoning processes, and information asymmetries, ML models pose inferential problems for them. These inferential problems lay the grounds for many ethical problems that currently claim centre-stage in the bioethical debate. Accordingly, this paper argues that the best way to make progress in remedying these ethical problems is to distil their epistemic core and to identify appropriate epistemic norms as guardrails. The viability of this approach will be highlighted based on four key issues: trust, responsibility, paternalism, and fairness. In that respect, the paper also contributes to a more pronounced view of the specific methodological challenges for ethical theorizing, when the point of reference are complex statistical models that make predictions, rather than individual agents, acting for reasons.

Space–time recurrent memory network

Transformers have recently been popular for learning and inference in the spatial–temporal domain. However, their performance relies on storing and applying attention to the feature tensor of each frame in video. Hence, their space and time complexity increase linearly as the length of video grows, which could be very costly for long videos. We propose a novel visual memory network architecture for the learning and inference problem in the spatial–temporal domain. We maintain a fixed set of memory slots in our memory network and propose an algorithm based on Gumbel-Softmax to learn an adaptive strategy to update this memory. Finally, this architecture is benchmarked on the video object segmentation (VOS) and video prediction problems. We demonstrate that our memory architecture achieves state-of-the-art results, outperforming transformer-based methods on VOS and other recent methods on video prediction while maintaining constant memory capacity independent of the sequence length.

Online Statistical Inference for Stochastic Optimization via Kiefer-Wolfowitz Methods

This article investigates the problem of online statistical inference of model parameters in stochastic optimization problems via the Kiefer-Wolfowitz algorithm with random search directions. We first present the asymptotic distribution for the Polyak-Ruppert-averaging type Kiefer-Wolfowitz (AKW) estimators, whose asymptotic covariance matrices depend on the distribution of search directions and the function-value query complexity. The distributional result reflects the tradeoff between statistical efficiency and function query complexity. We further analyze the choice of random search directions to minimize certain summary statistics of the asymptotic covariance matrix. Based on the asymptotic distribution, we conduct online statistical inference by providing two construction procedures of valid confidence intervals. Supplementary materials for this article are available online.

Evaluating evidence for co-geography in the Anopheles-Plasmodium host-parasite system.

The often tight association between parasites and their hosts means that under certain scenarios, the evolutionary histories of the two species can become closely coupled both through time and across space. Using spatial genetic inference, we identify a potential signal of common dispersal patterns in the Anopheles gambiae and Plasmodium falciparum host-parasite system as seen through a between-species correlation of the differences between geographic sampling location and geographic location predicted from the genome. This correlation may be due to coupled dispersal dynamics between host and parasite but may also reflect statistical artifacts due to uneven spatial distribution of sampling locations. Using continuous-space population genetics simulations, we investigate the degree to which uneven distribution of sampling locations leads to bias in prediction of spatial location from genetic data and implement methods to counter this effect. We demonstrate that while algorithmic bias presents a problem in inference from spatio-genetic data, the correlation structure between A. gambiae and P. falciparum predictions cannot be attributed to spatial bias alone and is thus likely a genetic signal of co-dispersal in a host-parasite system.

BenchMFC: A benchmark dataset for trustworthy malware family classification under concept drift

Concept drift poses a critical challenge in deploying machine learning models to mitigate practical malware threats. It refers to the phenomenon that the distribution of test data changes over time, gradually deviating from the original training data and degrading model performance. A promising direction for addressing concept drift is to detect drift samples and then retrain the model. However, this field currently lacks a unified, well-curated, and comprehensive benchmark, which often leads to unfair comparisons and inconclusive outcomes. To improve the evaluation and advance further, this paper presents a new Benchmark dataset for trustworthy Malware Family Classification (BenchMFC), which includes 223 K samples of 526 families that evolve over years. BenchMFC provides clear family, packer, and timestamp tags for each sample, it thus can support research on three types of malware concept drift: 1) unseen families, 2) packed families, and 3) evolved families. To collect unpacked family samples from large-scale candidates, we introduce a novel crowdsourcing malware annotation pipeline, which unifies packing detection and family annotation as a consensus inference problem to prevent costly packing detection. Moreover, we provide two case studies to illustrate the application of BenchMFC in 1) concept drift detection and 2) model retraining. The first case demonstrates the impact of three types of malware concept drift and compares nine notable concept drift detectors. The results show that existing detectors have their own advantages in dealing with different types of malware concept drift, and there is still room for improvement in malware concept drift detection. The second case explores how static feature-based machine learning operates on packed samples when retraining a model. The experiments illustrate that packers do preserve some kind of signals that appear to be “effective” for machine learning models, but the robustness of these signals requires further research. BenchMFC has been released to the community at https://github.com/crowdma/benchmfc.

Variance Estimation for Logistic Regression in Case-cohort Studies.

The logistic regression analysis proposed by Schouten et al (Stat Med. 1993;12:1733-1745) has been a standard method in current statistical analysis of case-cohort studies, and it enables effective estimation of risk ratios from selected subsamples, with adjustment of potential confounding factors. Schouten et al (1993) also proposed the standard error estimate of the risk ratio estimator can be calculated using the robust variance estimator, and this method has been widely adopted. The robust variance estimator does not account for the duplications of case and subcohort samples and generally has certain bias (ie, inaccurate confidence intervals and P-values are possibly obtained). To address the invalid statistical inference problem, we provide an alternative bootstrap-based valid variance estimator. Through simulation studies, the bootstrap method consistently provided more precise confidence intervals compared with those provided using the robust variance method, while retaining adequate coverage probabilities. The robust variance estimator has certain bias, and inadequate conclusions might be deduced from the resultant statistical analyses. The proposed bootstrap variance estimator can provide more accurate and precise interval estimates. The bootstrap method would be an alternative effective approach in practice to provide accurate evidence.

Robust Optimization and Inference on Manifolds

We propose a robust and scalable procedure for general optimization and inference problems on manifolds leveraging the classical idea of `median-of-means' estimation. This is motivated by ubiquitous examples and applications in modern data science in which a statistical learning problem can be cast as an optimization problem over manifolds. Being able to incorporate the underlying geometry for inference while addressing the need for robustness and scalability presents great challenges. We address these challenges by first proving a key lemma that characterizes some crucial properties of geometric medians on manifolds. In turn, this allows us to prove robustness and tighter concentration of our proposed final estimator in a subsequent theorem. This estimator aggregates a collection of subset estimators by taking their geometric median over the manifold. We illustrate bounds on this estimator via calculations in explicit examples. The robustness and scalability of the procedure is illustrated in numerical examples on both simulated and real data sets.

KID-PPG: Knowledge Informed Deep Learning for Extracting Heart Rate from a Smartwatch.

Accurate extraction of heart rate from photoplethysmography (PPG) signals remains challenging due to motion artifacts and signal degradation. Although deep learning methods trained as a data-driven inference problem offer promising solutions, they often underutilize existing knowledge from the medical and signal processing community. In this paper, we address three shortcomings of deep learning models: motion artifact removal, degradation assessment, and physiologically plausible analysis of the PPG signal. We propose KID-PPG, a knowledge-informed deep learning model that integrates expert knowledge through adaptive linear filtering, deep probabilistic inference, and data augmentation. We evaluate KID-PPG on the PPGDalia dataset, achieving an average mean absolute error of 2.85 beats per minute, surpassing existing reproducible methods. Our results demonstrate a significant performance improvement in heart rate tracking through the incorporation of prior knowledge into deep learning models. This approach shows promise in enhancing various biomedical applications by incorporating existing expert knowledge in deep learning models.

Application of fused graphical lasso to statistical inference for multiple sparse precision matrices.

In this paper, the fused graphical lasso (FGL) method is used to estimate multiple precision matrices from multiple populations simultaneously. The lasso penalty in the FGL model is a restraint on sparsity of precision matrices, and a moderate penalty on the two precision matrices from distinct groups restrains the similar structure across multiple groups. In high-dimensional settings, an oracle inequality is provided for FGL estimators, which is necessary to establish the central limit law. We not only focus on point estimation of a precision matrix, but also work on hypothesis testing for a linear combination of the entries of multiple precision matrices. We apply a de-biasing technology, which is used to obtain a new consistent estimator with known distribution for implementing the statistical inference, and extend the statistical inference problem to multiple populations. The corresponding de-biasing FGL estimator and its asymptotic theory are provided. A simulation study and an application of the diffuse large B-cell lymphoma data show that the proposed test works well in high-dimensional situation.

The efficacy and generalizability of conditional GANs for posterior inference in physics-based inverse problems

In this work, we train conditional Wasserstein generative adversarial networks to effectively sample from the posterior of physics-based Bayesian inference problems. The generator is constructed using a U-Net architecture, with the latent information injected using conditional instance normalization. The former facilitates a multiscale inverse map, while the latter enables the decoupling of the latent space dimension from the dimension of the measurement, and introduces stochasticity at all scales of the U-Net. We solve PDE-based inverse problems to demonstrate the performance of our approach in quantifying the uncertainty in the inferred field. Further, we show the generator can learn inverse maps which are local in nature, which in turn promotes generalizability when testing with out-of-distribution samples.