Likelihood Research Articles

Particle-Filter-Based State Estimation for Delayed Artificial Neural Networks: When Probabilistic Saturation Constraints Meet Redundant Channels.

In this brief, the state estimation problem is investigated for a class of randomly delayed artificial neural networks (ANNs) subject to probabilistic saturation constraints (PSCs) and non-Gaussian noises under the redundant communication channels. A series of mutually independent Bernoulli distributed white sequences are introduced to govern the random occurrence of the time delays, the saturation constraints, and the transmission channel failures. A comprehensive redundant-channel-based communication mechanism is constructed to attenuate the phenomenon of packet dropouts so as to enhance the quality of data transmission. To compensate for the influence of randomly occurring time delays, the corresponding occurrence probability is exploited in the process of particle generation. In addition, an explicit expression of the likelihood function is established based on the statistical information to account for the impact of PSCs and redundant channels. By virtue of the modified operations of particle propagation and weight update, a particle-filter-based state estimation algorithm is proposed with mild restriction on the system type. Finally, an illustrative example with Monte Carlo simulations is provided to demonstrate the effectiveness of the developed state estimation scheme.

Uncertainty quantification in logistic regression using random fuzzy sets and belief functions

Evidential likelihood-based inference is a new approach to statistical inference in which the relative likelihood function is interpreted as a possibility distribution. By expressing new data as a function of the parameter and a random variable with known probability distribution, one then defines a random fuzzy set and an associated predictive belief function representing uncertain knowledge about future observations. In this paper, this approach is applied to binomial and multinomial regression. In the binomial case, the predictive belief function can be computed by numerically integrating the possibility distribution of the posterior probability. In the multinomial case, the solution is obtained by a combination of constrained nonlinear optimization and Monte Carlo simulation. In both cases, computations can be considerably simplified using a normal approximation to the relative likelihood. Numerical experiments show that decision rules based on predictive belief functions make it possible to reach lower error rates for different rejection rates, as compared to decisions based on posterior probabilities.

Dentist: Quantifying uncertainty by sampling points around maximum likelihood estimates

Abstract It is standard statistical practice to provide measures of uncertainty around parameter estimates. Unfortunately, this very basic and necessary enterprise is often absent in macroevolutionary studies using maximum likelihood estimates (MLEs). dentist is an R package that allows an approximation of confidence intervals (CI) around parameter estimates without an analytic solution to likelihood equations. This package works by ‘denting’ the likelihood surface by sampling points a specified distance around the MLE following what is essentially a Metropolis‐Hastings walk. We describe the importance of estimating uncertainty around parameter estimates, as well as demonstrate the ability of dentist to accurately approximate CI. We introduce several plotting tools to visualize the results of a dentist analysis. dentist is freely available from https://github.com/bomeara/dentist, written in the R language, and can be used for any given likelihood function.

Methods and considerations for estimating parameters in biophysically detailed neural models with simulation based inference.

Biophysically detailed neural models are a powerful technique to study neural dynamics in health and disease with a growing number of established and openly available models. A major challenge in the use of such models is that parameter inference is an inherently difficult and unsolved problem. Identifying unique parameter distributions that can account for observed neural dynamics, and differences across experimental conditions, is essential to their meaningful use. Recently, simulation based inference (SBI) has been proposed as an approach to perform Bayesian inference to estimate parameters in detailed neural models. SBI overcomes the challenge of not having access to a likelihood function, which has severely limited inference methods in such models, by leveraging advances in deep learning to perform density estimation. While the substantial methodological advancements offered by SBI are promising, their use in large scale biophysically detailed models is challenging and methods for doing so have not been established, particularly when inferring parameters that can account for time series waveforms. We provide guidelines and considerations on how SBI can be applied to estimate time series waveforms in biophysically detailed neural models starting with a simplified example and extending to specific applications to common MEG/EEG waveforms using the the large scale neural modeling framework of the Human Neocortical Neurosolver. Specifically, we describe how to estimate and compare results from example oscillatory and event related potential simulations. We also describe how diagnostics can be used to assess the quality and uniqueness of the posterior estimates. The methods described provide a principled foundation to guide future applications of SBI in a wide variety of applications that use detailed models to study neural dynamics.

A facies‐constrained geostatistical seismic inversion method based on multi‐scale sparse representation

AbstractGeostatistical seismic inversion is an important method for establishing high‐resolution reservoir parameter models. There is no accurate representation method for reservoir structural features, and prior information about structural features cannot be incorporated into geostatistical inversion. Based on the assumption of the sparsity of stratigraphic sedimentary features, the same type of structural feature is used to represent the sedimentary pattern of reservoirs within the same facies. Different sparse representation patterns are used to represent the differences in sedimentary patterns between facies. Although changes in depositional environment might result in the multi‐scale characteristics of geological structures for varying sedimentary rhythms, this paper proposes a facies‐constrained geostatistical inversion method based on multi‐scale sparse representation to better accommodate such situation. Using the method of sparse representation combined with wavelet transform, the multi‐scale sedimentary structural features of reservoirs are learned from well‐logging data. Seismic facies and multi‐scale features are used as prior information for geostatistical inversion. Further, the likelihood function is constructed using seismic data to obtain the posterior probability distribution of reservoir parameters. Finally, the accurate inversion result is obtained by using multi‐scale sparse representation as a constraint in the posterior probability distribution of reservoir parameters. Compared with conventional geostatistical methods, this algorithm can better match the structural features of reservoir parameters with varying geological conditions. Field data tests have shown the effectiveness of this method in improving the accuracy and resolution of reservoir parameter structural features.

A Technique for Efficient Estimation of Dynamic Structural Equation Models: A Case Study

Dynamic structural equation models (DSEM) are designed for time series analysis of latent structures. Inherent to the application of DSEM is model parameter estimation, which has to be addressed in many applications by a single time series. In this context, however, the methods currently available either lack estimation quality or are computationally inefficient. Given the era of big data, the necessity for a trade-off between these properties may be detrimental to the applicability of DSEM. The paper is aimed at tackling this trade-off by proposing a novel estimator recursioning technique (ER technique) that facilitates the development of computationally efficient raw-data maximum likelihood estimation algorithms through data transformation, covariance matrix block decomposition, likelihood function reduction, and estimator recursioning steps. The ER technique is introduced by applying it to a special case of the general dynamic structural equation model that encompasses a noisy Wiener-process-type structure with input from the factor-analytic model. The resulting algorithm has been verified through a number of numerical experiments as well as implemented in a brand new R package EMLI, which is available on CRAN.

Incorporating Ocean-Borne Debris Information into Search Object Location Distributions

This paper presents a method for incorporating information from waterborne debris that has been recovered at a known time and place into the object location distribution. The method involves calculating a likelihood function which can be combined with the prior distribution on the location of a lost object, such as a ship or aircraft lost at sea, to produce, in a Bayesian fashion, a posterior distribution that reduces uncertainty in the location of the loss. When the search operation involves a person or vessel adrift at sea, incorporating debris information in this fashion will allow the search planner to produce better predictions for the location of the search object at times of future searches and enable more effective deployment of search assets. Use of this method will save time, money, and lives. KEY WORDS: Maritime Search and Rescue, Ocean-Borne Debris, Maritime Loss, Debris Drift, Object Location Distribution, Bayesian, Posterior Distribution

Selection among site-dependent structurally constrained substitution models of protein evolution by approximate Bayesian computation.

The selection among substitution models of molecular evolution is fundamental for obtaining accurate phylogenetic inferences. At the protein level, evolutionary analyses are traditionally based on empirical substitution models but these models make unrealistic assumptions and are being surpassed by structurally constrained substitution (SCS) models. The SCS models often consider site-dependent evolution, a process that provides realism but complicates their implementation into likelihood functions that are commonly used for substitution model selection. We present a method to perform selection among site-dependent SCS models, also among empirical and site-dependent SCS models, based on the approximate Bayesian computation (ABC) approach and its implementation into the computational framework ProteinModelerABC. The framework implements ABC with and without regression adjustments and includes diverse empirical and site-dependent SCS models of protein evolution. Using extensive simulated data, we found that it provides selection among SCS and empirical models with acceptable accuracy. As illustrative examples, we applied the framework to analyze a variety of protein families observing that SCS models fit them better than the corresponding best-fitting empirical substitution models. ProteinModelerABC is freely available from https://github.com/DavidFerreiro/ProteinModelerABC, can run in parallel and includes a graphical user interface. The framework is distributed with detailed documentation and ready-to-use examples.

Swarm Intelligence Methods for Extreme Mass Ratio Inspiral Search: First Application of Particle Swarm Optimization

Swarm intelligence (SI) methods are nature-inspired metaheuristics for global optimization that exploit a coordinated stochastic search strategy by a group of agents. Particle swarm optimization (PSO) is an established SI method that has been applied successfully to the optimization of rugged high-dimensional likelihood functions, a problem that represents the main bottleneck across a variety of gravitational wave (GW) data analysis challenges. We present results from the first application of PSO to one of the most difficult of these challenges, namely the search for the Extreme Mass Ratio Inspiral (EMRI) in data from future spaceborne GW detectors such as LISA, Taiji, or Tianqin. We use the standard Generalized Likelihood Ratio Test formalism, with the minimal use of restrictive approximations, to search 6 months of simulated LISA data and quantify the search depth, signal-to-noise ratio (SNR), and breadth, within the ranges of the EMRI parameters, that PSO can handle. Our results demonstrate that a PSO-based EMRI search is successful for a search region ranging over ≳10σ for the majority of parameters and ≳200σ for one, with σ being the SNR-dependent Cramer–Rao lower bound on the parameter estimation error and 30≤SNR≤50. This is in the vicinity of the search ranges that the current hierarchical schemes can identify. Directions for future improvement, including computational bottlenecks to be overcome, are identified.

Bayesian model calibration for block copolymer self-assembly: Likelihood-free inference and expected information gain computation via measure transport

We consider the Bayesian calibration of models describing the phenomenon of block copolymer (BCP) self-assembly, which is to infer model parameters and their uncertainty from noisy image data produced by microscopy or X-ray scattering characterization of BCP structures. To account for the long-range disorder in BCP structures, we introduce auxiliary variables in the model to represent this aleatory uncertainty. These variables, however, result in an integrated likelihood function for high-dimensional image data that is generally intractable to evaluate. We tackle this Bayesian inference problem using a likelihood-free inference (LFI) approach based on measure transport together with the construction of summary statistics for the image data. We show that expected information gains (EIG) from observed data about the model parameters can be computed with no significant additional cost. Lastly, we present a numerical case study using the Ohta–Kawasaki model for diblock copolymer thin film self-assembly and top-down microscopy characterization. We introduce several domain-specific energy and Fourier-based summary statistics for calibration and quantify their informativeness using EIG. The effect of various data corruptions, summary statistics, and experimental designs on the calibration results are studied using the proposed LFI method.

Quantum Tomography: From Markovianity to Non-Markovianity

The engineering of quantum computers requires the reliable characterization of qubits, quantum operations, and even the entire hardware. Quantum tomography is an indispensable framework in quantum characterization, verification, and validation (QCVV), which has been widely accepted by researchers. According to the tomographic target, quantum tomography can be categorized into quantum state tomography (QST), quantum process tomography (QPT), gate set tomography (GST), process tensor tomography (PTT), and instrument set tomography (IST). Standard quantum tomography toolkits generally consist of basic linear inverse methods and statistical maximum likelihood estimation (MLE)-based methods. Furthermore, the performance of standard methods, including effectiveness and efficiency, has been further developed by exploiting Bayesian estimation, neural networks, matrix completion techniques, etc. In this review, we introduce the fundamental quantum tomography techniques, including QST, QPT, GST, PTT, and IST. We first introduce the details of basic linear inverse methods. Then, the framework of MLE methods with constraints is summarized. Finally, we briefly introduce recent further research in developing the performance of tomography, utilizing some symmetry properties of the target. This review provides a primary getting-start in developing quantum tomography, which promotes quantum computer development.

Asymptotic Form of the Covariance Matrix of Likelihood-Based Estimator in Multidimensional Linear System Model for the Case of Infinity Number of Nuisance Parameters

This article is devoted to the synthesis and analysis of the quality of the statistical estimate of parameters of a multidimensional linear system (MLS) with one input and m outputs. A nontrivial case is investigated when the one-dimensional input signal of MLS is a deterministic process, the values of which are unknown nuisance parameters. The estimate is based only on observations of MLS output signals distorted by random Gaussian stationary m-dimensional noise with a known spectrum. It is assumed that the likelihood function of observations of the output signals of MLS satisfies the conditions of local asymptotic normality. The n-consistency of the estimate is established. Under the assumption of asymptotic normality of an objective function, the limiting covariance matrix of the estimate is calculated for case where the number of observations tends to infinity.

A longitudinal complex likelihood ratio test for pleiotropy

. To test pleiotropy based on longitudinal data, two main difficulties are the construction of test statistics and the formulation of decision-making criteria. In particular, the commonly used log-likelihood ratio test is not available since there does not exist a likelihood function and the estimation is based on estimating equations in general. To address this and construct a statistic similar to the -2 log-likelihood ratio test statistic, we propose a new concept, a complex log-likelihood function, such that its first derivative is equal to a given estimating function. In consequence, the -2 complex log-likelihood ratio test is constructed for testing genetic pleiotropy, and the validity of the test procedure is established. An extensive simulation study is performed, which suggests that the proposed method works well for practical situations. Furthermore, it is applied to a set of longitudinal data on tropical storms.

Updating information based on generalized credal sets. Part 1: Finite case

This paper considers updating information and statistical inference based on generalized credal sets. In the first part, we define possible ways of conditioning for generalized credal sets in a finite setting, and its realization based on generalized upper previsions with applications to decision making. We compare the obtained results with the generalized Bayes rule from the theory of imprecise probabilities, with Dempster's rule from the theory of belief functions, and with the models of imprecise probabilities based on the likelihood function proposed by M. Cattaneo.

Likelihood ratio combination of multiple biomarkers viasmoothing spline estimated densities.

The diagnostic accuracy of multiple biomarkers in medical research is crucial for detecting diseases and predicting patient outcomes. An optimal method for combining these biomarkers is essential to maximize the Area Under the Receiver Operating Characteristic (ROC) Curve (AUC). Although the optimality of the likelihood ratio has been proven by Neyman and Pearson, challenges persist in estimating the likelihood ratio, primarily due to the estimation of multivariate density functions. In this study, we propose a non-parametric approach for estimating multivariate density functions by utilizing Smoothing Spline density estimation to approximate the full likelihood function for both diseased and non-diseased groups, which compose the likelihood ratio. Simulation results demonstrate the efficiency of our method compared to other biomarker combination techniques under various settings for generated biomarker values. Additionally, we apply the proposed method to a real-world study aimed at detecting childhood autism spectrum disorder (ASD), showcasing its practical relevance and potential for future applications in medical research.

VB-Based Gaussian Sum Cubature Kalman Filter for Adaptive Estimation of Unknown Delay and Loss Probability

The traditional Kalman filter assumes that all measurements can be obtained in real time, which is invalid in practical engineering. Therefore, a variational Bayesian- (VB-) based Gaussian sum cubature Kalman filter is proposed to solve the nonlinear tracking problem of multistep random measurement delay and loss (MRMDL) with unknown probability. First, the measurement model with MRMDL is modified by Bernoulli random variables. Then, the expression of the likelihood function is reformulated as a mixture of multiple Gaussian distributions, and the cubature rule is used to improve the estimation accuracy under the framework of Gaussian sum filter in the process of time update. Finally, by constructing a hierarchical Gaussian model, the unknown and time-varying measurement delay and loss probability are estimated in real time with the state jointly using the VB method in the measurement update stage. The algorithm does not need to calculate the equivalent noise covariance matrix so as to avoid the possible division by zero operation, which improves the stability of the algorithm. Simulation results for a target tracking problem show that the proposed algorithm has a better performance in the presence of MRMDL and can estimate the unknown measurement delay and loss probability accurately.

Considerations in Bayesian agent‐based modeling for the analysis of COVID‐19 data

AbstractAgent‐based model (ABM) has been widely used to study infectious disease transmission by simulating behaviors and interactions of autonomous individuals called agents. In the ABM, agent states, for example infected or susceptible, are assigned according to a set of simple rules, and a complex dynamics of disease transmission is described by the collective states of agents over time. Despite the flexibility in real‐world modeling, ABMs have received less attention by statisticians because of the intractable likelihood functions which lead to difficulty in estimating parameters and quantifying uncertainty around model outputs. To overcome this limitation, a Bayesian framework that treats the entire ABM as a Hidden Markov Model has been previously proposed. However, existing approach is limited due to computational inefficiency and unidentifiability of parameters. We extend the ABM approach within Bayesian framework to study infectious disease transmission addressing these limitations. We estimate the hidden states, represented by individual agent's states over time, and the model parameters by applying an improved particle Markov Chain Monte Carlo algorithm, that accounts for computing efficiency. We further evaluate the performance of the approach for parameter recovery and prediction, along with sensitivity to prior assumptions under various simulation conditions. Finally, we apply the proposed approach to the study of COVID‐19 outbreak on Diamond Princess cruise ship. We examine the differences in transmission by key demographic characteristics, while considering two different networks and limited COVID‐19 testing in the cruise.

Inferring the Goals of Communicating Agents from Actions and Instructions

When humans cooperate, they frequently coordinate their activity through both verbal communication and non-verbal actions, using this information to infer a shared goal and plan. How can we model this inferential ability? In this paper, we introduce a model of a cooperative team where one agent, the principal, may communicate natural language instructions about their shared plan to another agent, the assistant, using GPT-3 as a likelihood function for instruction utterances. We then show how a third person observer can infer the team’s goal via multi-modal Bayesian inverse planning from actions and instructions, computing the posterior distribution over goals under the assumption that agents will act and communicate rationally to achieve them. We evaluate this approach by comparing it with human goal inferences in a multi-agent gridworld, finding that our model’s inferences closely correlate with human judgments (R = 0.96). When compared to inference from actions alone, we find that instructions lead to more rapid and less uncertain goal inference, highlighting the importance of verbal communication for cooperative agents.

Multivariate adaptive regression splines-assisted approximate Bayesian computation for calibration of complex hydrological models

Abstract Approximate Bayesian computation (ABC) relaxes the need to derive explicit likelihood functions required by formal Bayesian analysis. However, the high computational cost of evaluating models limits the application of Bayesian inference in hydrological modeling. In this paper, multivariate adaptive regression splines (MARS) are used to expedite the ABC calibration process. The MARS model is trained using 6,561 runoff simulations generated by the soil and water assessment tool (SWAT) model and subsequently replaces the SWAT model to calculate the objective functions in ABC and multi-objective evolutionary algorithm (MOEA). In experiments, MARS can successfully reproduce the runoff time series simulations of the SWAT model at a low time cost, with a runoff variance determination coefficient of 0.90 as compared to the Monte Carlo method. MARS-assisted ABC can quickly and accurately estimate the parameter distributions of the SWAT model. The comparison of ABC with non-Bayesian MOEAs helps in the selection of an appropriate calibration approach.

Multiomic analysis for minimal residual disease detection: Addressing challenges in stage II-III colon cancer from COSMOS-CRC-01.

180 Background: A considerable number of patients with stage II-III colon cancer experience disease recurrence following definitive treatment while others may receive unnecessary toxicity from adjuvant chemotherapy. Minimal residual disease (MRD) detection through ctDNA analysis can be evaluated without reliance on tissue which comes with logistic advantages and may be the only option for patients with limited tumor specimens. Methods: COSMOS-CRC-01 is an ongoing biomarker study that enrolled patients with resectable stage 0-III colorectal cancer between January 2020 and April 2021. Plasma samples and clinical data were collected 28 days post-operatively and every 3-6 months for up to 5 years alongside radiographic imaging. Samples were tested on the Guardant Reveal Infinity Oncology Platform for MRD detection. This analytically validated multiomic NGS assay interrogates >700 genes and ~15Mb of methylated regions. A bioinformatics algorithm trained for CRC detection classifies each sample as ctDNA detectable or undetectable based on a predefined statistical likelihood threshold and returns quantitative tumor fraction. The primary endpoint was time to recurrence (TTR) based on ctDNA detection status. Results: This subgroup analysis evaluated 801 post-surgical plasma samples (4mL each, median input 19ng, 96% passed QC) from 136 patients with R0 resected stage II-III colon cancer with a median follow-up of 28 months. ctDNA detection at Day 28 post-surgery and following adjuvant chemotherapy were both associated with significantly shorter TTR (p<0.0001; Table). Sensitivity with serial measurement was 80% (95% CI, 56.3-94.3%), with a median lead time of 5.3 months. Sensitivity was highest for detection of locoregional (2/2), brain (1/1), liver (7/7), or multiple (3/3) metastasis compared to isolated lung (2/4) or peritoneal (1/3) metastasis. Tumor fraction increased as expected in samples collected closer to the time of radiographic detection. Three recurred patients had KRAS G12C variants detected in ctDNA prior to recurrence, suggesting the potential of early intervention with targeted agents. Among 553 post-treatment samples from 114 non-recurred patients, sample specificity was 97.3% (95% CI 95.6-98.5%). Conclusions: We show a plasma-only genomic- and methylation-based assay to have sensitive and specific detection of MRD in stage II-III colon cancer. Serial measurement provides superior performance versus one-time measurement at Day 28 post-surgery and detects targetable variants prior to recurrence. [Table: see text]