Gaussian Likelihood Research Articles

The effect on cosmological parameter estimation of a parameter dependent covariance matrix

Cosmological large-scale structure analyses based on two-point correlation functions often assume a Gaussian likelihood function with a fixed covariance matrix. We study the impact on cosmological parameter estimation of ignoring the parameter dependence of this covariance matrix, focusing on the particular case of joint weak-lensing and galaxy clustering analyses. Using a Fisher matrix formalism (calibrated against exact likelihood evaluation in particular simple cases), we quantify the effect of using a parameter dependent covariance matrix on both the bias and variance of the parameters. We confirm that the approximation of a parameter-independent covariance matrix is exceptionally good in all realistic scenarios. The information content in the covariance matrix (in comparison with the two point functions themselves) does not change with the fractional sky coverage. Therefore the increase in information due to the parameter dependent covariance matrix becomes negligible as the number of modes increases. Even for surveys covering less than 1% of the sky, this effect only causes a bias of up to of order 10% of the statistical uncertainties, with a misestimation of the parameter uncertainties at the same level or lower. The effect will only be smaller with future large-area surveys. Thus for most analyses the effect of a parameter-dependent covariance matrix can be ignored both in terms of the accuracy and precision of the recovered cosmological constraints.

Cosmological inference from galaxy-clustering power spectrum: Gaussianization and covariance decomposition

Likelihood fitting to two-point clustering statistics made from galaxy surveys usually assumes a multivariate normal distribution for the measurements, with justification based on the central limit theorem given the large number of overdensity modes. However, this assumption cannot hold on the largest scales where the number of modes is low. Whilst more accurate distributions have previously been developed in idealized cases, we derive a procedure suitable for analysing measured monopole power spectra with window effects, stochastic shot noise, and the dependence of the covariance matrix on the model being fitted all taken into account. A data transformation is proposed to give an approximately Gaussian likelihood, with a variance--correlation decomposition of the covariance matrix to account for its cosmological dependence. By comparing with the modified-$t$ likelihood derived under the usual normality assumption, we find in numerical tests that our new procedure gives more accurate constraints on the local non-Gaussianity parameter $f_\mathrm{NL}$, which is sensitive to the large-scale power. A simple data analysis pipeline is provided for straightforward application of this new approach in preparation for forthcoming large galaxy surveys such as DESI and Euclid.

Computation of the maximum likelihood estimator in low-rank factor analysis

Factor analysis is a classical multivariate dimensionality reduction technique popularly used in statistics, econometrics and data science. Estimation for factor analysis is often carried out via the maximum likelihood principle, which seeks to maximize the Gaussian likelihood under the assumption that the positive definite covariance matrix can be decomposed as the sum of a low-rank positive semidefinite matrix and a diagonal matrix with nonnegative entries. This leads to a challenging rank constrained nonconvex optimization problem, for which very few reliable computational algorithms are available. We reformulate the low-rank maximum likelihood factor analysis task as a nonlinear nonsmooth semidefinite optimization problem, study various structural properties of this reformulation; and propose fast and scalable algorithms based on difference of convex optimization. Our approach has computational guarantees, gracefully scales to large problems, is applicable to situations where the sample covariance matrix is rank deficient and adapts to variants of the maximum likelihood problem with additional constraints on the model parameters. Our numerical experiments validate the usefulness of our approach over existing state-of-the-art approaches for maximum likelihood factor analysis.

A Bayesian race model for response times under cyclic stimulus discriminability

Response time (RT) data from psychology experiments are often used to validate theories of how the brain processes information and how long it takes a person to make a decision. When an RT results from a task involving two or more possible responses, the cognitive process that determines the RT may be modeled as the first-passage time of underlying competing (racing) processes with each process describing accumulation of information in favor of one of the responses. In one popular model the racers are assumed to be Gaussian diffusions. Their first-passage times are inverse Gaussian random variables and the resulting RT has a min-inverse Gaussian distribution. The RT data analyzed in this paper were collected in an experiment requiring people to perform a two-choice task in response to a regularly repeating sequence of stimuli. Starting from a min-inverse Gaussian likelihood for the RTs we build a Bayesian hierarchy for the rates and thresholds of the racing diffusions. The analysis allows us to characterize patterns in a person’s sequence of responses on the basis of features of the person’s diffusion rates (the “footprint” of the stimuli) and a person’s gradual changes in speed as trends in the diffusion thresholds. Last, we propose that a small fraction of RTs arise from distinct, noncognitive processes that are included as components of a mixture model. In the absence of sharp prior information, the inclusion of these mixture components is accomplished via a two-stage, empirical Bayes approach. The resulting framework may be generalized readily to RTs collected under a variety of experimental designs.

A Bayesian model for lithology/fluid class prediction using a Markov mesh prior fitted from a training image

ABSTRACTWe consider a Bayesian model for inversion of observed amplitude variation with offset data into lithology/fluid classes, and study in particular how the choice of prior distribution for the lithology/fluid classes influences the inversion results. Two distinct prior distributions are considered, a simple manually specified Markov random field prior with a first‐order neighbourhood and a Markov mesh model with a much larger neighbourhood estimated from a training image. They are chosen to model both horizontal connectivity and vertical thickness distribution of the lithology/fluid classes, and are compared on an offshore clastic oil reservoir in the North Sea. We combine both priors with the same linearized Gaussian likelihood function based on a convolved linearized Zoeppritz relation and estimate properties of the resulting two posterior distributions by simulating from these distributions with the Metropolis–Hastings algorithm. The influence of the prior on the marginal posterior probabilities for the lithology/fluid classes is clearly observable, but modest. The importance of the prior on the connectivity properties in the posterior realizations, however, is much stronger. The larger neighbourhood of the Markov mesh prior enables it to identify and model connectivity and curvature much better than what can be done by the first‐order neighbourhood Markov random field prior. As a result, we conclude that the posterior realizations based on the Markov mesh prior appear with much higher lateral connectivity, which is geologically plausible.

Bootstrapping impulse responses of structural vector autoregressive models identified through GARCH

Different bootstrap methods and estimation techniques for inference for structural vector autoregressive (SVAR) models identified by generalized autoregressive conditional heteroskedasticity (GARCH) are reviewed and compared in a Monte Carlo study. The bootstrap methods considered are a wild bootstrap, a moving blocks bootstrap and a GARCH residual based bootstrap. Estimation is done by Gaussian maximum likelihood, a simplified procedure based on univariate GARCH estimations and a method that does not re-estimate the GARCH parameters in each bootstrap replication. The latter method is computationally more efficient than the other methods and it is competitive with the other methods and often leads to the smallest confidence sets without sacrificing coverage precision. An empirical model for assessing monetary policy in the U.S. is considered as an example. It is found that the different inference methods for impulse responses lead to qualitatively very similar results.

Joint Power Spectrum and Voxel Intensity Distribution Forecast on the CO Luminosity Function with COMAP

Abstract We develop a framework for joint constraints on the CO luminosity function based on power spectra (PS) and voxel intensity distributions (VID) and apply this to simulations of CO Mapping Array Pathfinder (COMAP), a CO intensity mapping experiment. This Bayesian framework is based on a Markov chain Monte Carlo (MCMC) sampler coupled to a Gaussian likelihood with a joint PS + VID covariance matrix computed from a large number of fiducial simulations and re-calibrated with a small number of simulations per MCMC step. The simulations are based on dark matter halos from fast peak patch simulations combined with the L CO(M halo) model of Li et al. We find that the relative power to constrain the CO luminosity function depends on the luminosity range of interest. In particular, the VID is more sensitive at large luminosities, while the PS and the VID are both competitive at small and intermediate luminosities. The joint analysis is superior to using either observable separately. When averaging over CO luminosities ranging between , and over 10 cosmological realizations of COMAP Phase 2, the uncertainties (in dex) are larger by 58% and 30% for the PS and VID, respectively, when compared to the joint analysis (PS + VID). This method is generally applicable to any other random field, with a complicated likelihood, as long a fast simulation procedure is available.

A variable high-order shock-capturing finite difference method with GP-WENO

We present a new finite difference shock-capturing scheme for hyperbolic equations on static uniform grids. The method provides selectable high-order accuracy by employing a kernel-based Gaussian Process (GP) data prediction method which is an extension of the GP high-order method originally introduced in a finite volume framework by the same authors. The method interpolates Riemann states to high order, replacing the conventional polynomial interpolations with polynomial-free GP-based interpolations. For shocks and discontinuities, this GP interpolation scheme uses a nonlinear shock handling strategy similar to Weighted Essentially Non-Oscillatory (WENO), with a novelty consisting in the fact that nonlinear smoothness indicators are formulated in terms of the Gaussian likelihood of the local stencil data, replacing the conventional L2-type smoothness indicators of the original WENO method. We demonstrate that these GP-based smoothness indicators play a key role in the new algorithm, providing significant improvements in delivering high – and selectable – order accuracy in smooth flows, while successfully delivering non-oscillatory solution behavior in discontinuous flows.

A new measure of tension between experiments

Tensions between cosmological measurements by different surveys or probes have always been important —and are presently much discussed —as they may lead to evidence of new physics. Several tests have been devised to probe the consistency of datasets given a cosmological model, but they often have undesired features such as dependence on the prior volume, or burdensome requirements such as that of near-Gaussian posterior distributions. We propose a new quantity, defined in a similar way as the Bayesian evidence ratio, in which these undesired properties are absent. We test the quantity on simple models with Gaussian and non-Gaussian likelihoods. We then apply it to data from the Planck satellite: we investigate the consistency of ΛCDM model parameters obtained from TT and EE angular power spectrum measurements, as well as the mutual consistency of cosmological parameters obtained from large scale (multipoles, ℓ<1000) and small scale (ℓ ⩾ 1000) portions of each measurement and find no significant discrepancy in the six-dimensional ΛCDM parameter space.

Large-scale Sparse Inverse Covariance Matrix Estimation

The estimation of large sparse inverse covariance matrices is a ubiquitous statistical problem in many application areas such as mathematical finance, geology, health, and many others. The $\ell_1$-regularized Gaussian maximum likelihood (ML) method is a common approach for recovering inverse covariance matrices for datasets with a very limited number of samples. A highly efficient ML-based method is the quadratic approximate inverse covariance (QUIC) method. In this work, we build on the advancements of QUIC algorithm by introducing a highly performant sparse version of QUIC (SQUIC) for large-scale applications. The proposed algorithm focuses on exploiting the potential sparsity in three components of the QUIC algorithm, namely, construction sample covariance matrix, matrix factorization, and matrix inversion operations. For each component, we present two approaches and provide supporting numerical results based on a set of synthetic datasets and a stylized financial autoregressive model. Testing conducted on a single modern multicore machine show that using advanced sparse matrix technology, SQUIC can recover large-scale inverse covariance matrices of datasets with up to $1$ million random variables within minutes. In comparison to competing ML-based algorithms, SQUIC is orders of magnitude faster with comparable recovery rates.

Forensic Gender Speaker Recognition under Clean and Noisy Environments

The Forensic Gender Speaker Recognition (FGSR) deals with finding out the gender of a person from his or her voice. This task has been implemented in several Automatic Speaker Recognition systems and has proved to be of great significance. The use of gender recognition eases for user authentication and identification in high security systems. In this paper, we have used the gender recognition for speakers using Mel Frequency Cepstral Coefficients (MFCCs), a Gaussian Mixture Model-Universal Background Model (GMM-UBM) and Likelihood Report (LR) were used for modelization of features extracted from the speech signal of speakers under clean and noisy conditions. The performance of FGSR system is evaluated in terms of Equal Proportion Probability (EPP). Three sets of experiments were performed - the first experiment; corpus contains males’ voices. The second; contains females’ voices and third; contains both voices of both sexes. In clean conditions, the average accuracy of the first experiment was slightly higher for EPP=3.1579% than the second and the third experiments. Therefore, Performance evaluation results are encouraging in clean conditions compared to in noisy condition for EPP=27%.

WASSERSTEIN METRIC-DRIVEN BAYESIAN INVERSION WITH APPLICATIONS TO SIGNAL PROCESSING

We present a Bayesian framework based on a new exponential likelihood function driven by the quadratic Wasserstien metric. Compared to conventional Bayesian models based on Gaussian likelihood functions driven by the least-squares norm ($L_2$ norm), the new framework features several advantages. First, the new framework does not rely on the likelihood of the measurement noise and hence can treat complicated noise structures such as combined additive and multiplicative noise. Secondly, unlike the normal likelihood function, the Wasserstein-based exponential likelihood function does not usually generate multiple local extrema. As a result, the new framework features better convergence to correct posteriors when a Markov Chain Monte Carlo sampling algorithm is employed. Thirdly, in the particular case of signal processing problems, while a normal likelihood function measures only the amplitude differences between the observed and simulated signals, the new likelihood function can capture both the amplitude and the phase differences. We apply the new framework to a class of signal processing problems, that is, the inverse uncertainty quantification of waveforms, and demonstrate its advantages compared to Bayesian models with normal likelihood functions.

Detecting Co‐Movements in Non‐Causal Time Series

AbstractThis paper introduces the notion of common non‐causal features and proposes tools to detect them in multivariate time series models. We argue that the existence of co‐movements might not be detected using the conventional stationary vector autoregressive (VAR) model as the common dynamics are present in the non‐causal (i.e. forward‐looking) component of the series. We show that the presence of a reduced rank structure allows to identify purely causal and non‐causal VAR processes of order P&gt;1 even in the Gaussian likelihood framework. Hence, usual test statistics and canonical correlation analysis can be applied, where either lags or leads are used as instruments to determine whether the common features are present in either the backward‐ or forward‐looking dynamics of the series. The proposed definitions of co‐movements are also valid for the mixed causal—non‐causal VAR, with the exception that a non‐Gaussian maximum likelihood estimator is necessary. This means however that one loses the benefits of the simple tools proposed. An empirical analysis on Brent and West Texas Intermediate oil prices illustrates the findings. No short run co‐movements are found in a conventional causal VAR, but they are detected when considering a purely non‐causal VAR.

Representation, estimation and forecasting of the multivariate index-augmented autoregressive model

We examine the conditions under which each individual series that is generated by a vector autoregressive model can be represented as an autoregressive model that is augmented with the lags of a few linear combinations of all the variables in the system. We call this multivariate index-augmented autoregression (MIAAR) modelling. We show that the parameters of the MIAAR can be estimated by a switching algorithm that increases the Gaussian likelihood at each iteration. Since maximum likelihood estimation may perform poorly when the number of parameters increases, we propose a regularized version of our algorithm for handling a medium–large number of time series. We illustrate the usefulness of the MIAAR modelling by both empirical applications and simulations.

Bayesian Modeling of Motion Perception Using Dynamical Stochastic Textures.

A common practice to account for psychophysical biases in vision is to frame them as consequences of a dynamic process relying on optimal inference with respect to a generative model. The study presented here details the complete formulation of such a generative model intended to probe visual motion perception with a dynamic texture model. It is derived in a set of axiomatic steps constrained by biological plausibility. We extend previous contributions by detailing three equivalent formulations of this texture model. First, the composite dynamic textures are constructed by the random aggregation of warped patterns, which can be viewed as three-dimensional gaussian fields. Second, these textures are cast as solutions to a stochastic partial differential equation (sPDE). This essential step enables real-time, on-the-fly texture synthesis using time-discretized autoregressive processes. It also allows for the derivation of a local motion-energy model, which corresponds to the log likelihood of the probability density. The log likelihoods are essential for the construction of a Bayesian inference framework. We use the dynamic texture model to psychophysically probe speed perception in humans using zoom-like changes in the spatial frequency content of the stimulus. The human data replicate previous findings showing perceived speed to be positively biased by spatial frequency increments. A Bayesian observer who combines a gaussian likelihood centered at the true speed and a spatial frequency dependent width with a "slow-speed prior" successfully accounts for the perceptual bias. More precisely, the bias arises from a decrease in the observer's likelihood width estimated from the experiments as the spatial frequency increases. Such a trend is compatible with the trend of the dynamic texture likelihood width.

Bayesian sparse signal recovery based on log-Laplacian prior

This paper proposes a Bayesian sparse signal recovery algorithm. To improve performance on sparse representation, the log-Laplacian distribution is first defined. With a narrow main lobe and high tail values, it is used as a prior of the sparse signal to model the sparse characteristic. Note that analytical inference of the posterior of the sparse signal is a challenge, because the proposed log-Laplacian prior is not conjugate to the Gaussian likelihood. A maximum a posterior (MAP) estimation-based sparse signal recovery algorithm is further proposed. During the reconstruction of the sparse signal, MAP and maximum likelihood estimation are utilized to estimate the scaling parameter and noise variance, respectively, so as to avoid manual tuning of parameters. Additionally, with the use of the conjugate-gradient algorithm, large matrix inversion is avoided and computational efficiency is improved. Experimental results based on both simulated and measured data validate the effectiveness of the proposed log-Laplacian prior-based sparse signal recovery algorithm. Further, it is applied to micromotion parameters estimation and inverse synthetic aperture radar imaging to confirm its validity.

Cone penetration data classification by Bayesian inversion with a Hidden Markov model

This study examines the application of the Hidden Markov model (HMM) to the soil classification based on Cone Penetration Test (CPT) measurements. The HMM is formulated in the Bayesian framework and composed of a Markov chain prior and a Gaussian likelihood model. The application of the Bayesian framework is considered as suitable because it allows for the integration of different sources of information commonly available in a CPT-based soil classification. The occurrence of different soil classes along a CPT profile is modeled with the Markov chain, while the Gaussian likelihood model establishes a relation between the different soil classes and CPT measurements. Preliminary performance of the HMM is examined on the classification of CPT measurements from the Sheringham Shoal Offshore Wind Farm.

Estimation of Urban Link Travel Time Distribution Using Markov Chains and Bayesian Approaches

Despite the wide application of Floating Car Data (FCD) in urban link travel time and congestion estimation, the sparsity of observations from a low penetration rate of GPS-equipped floating cars make it difficult to estimate travel time distribution (TTD), especially when the travel times may have multimodal distributions that are associated with the underlying traffic states. In this case, the study develops a Bayesian approach based on particle filter framework for link TTD estimation using real-time and historical travel time observations from FCD. First, link travel times are classified by different traffic states according to the levels of vehicle delays. Then, a state-transition function is represented as a Transition Probability Matrix of the Markov chain between upstream and current links with historical observations. Using the state-transition function, an importance distribution is constructed as the summation of historical link TTDs conditional on states weighted by the current link state probabilities. Further, a sampling strategy is developed to address the sparsity problem of observations by selecting the particles with larger weights in terms of the importance distribution and a Gaussian likelihood function. Finally, the current link TTD can be reconstructed by a generic Markov Chain Monte Carlo algorithm incorporating high weighted particles. The proposed approach is evaluated using real-world FCD. The results indicate that the proposed approach provides good accurate estimations, which are very close to the empirical distributions. In addition, the approach with different percentage of floating cars is tested. The results are encouraging, even when multimodal distributions and very few or no observations exist.

Likelihood-free inference in high dimensions with synthetic likelihood

One popular approach to likelihood-free inference is the synthetic likelihood method, which assumes that some data summary statistics which are informative about model parameters are approximately Gaussian for each value of the parameter. Based on this assumption, a Gaussian likelihood can be constructed, where the mean and covariance matrix of the summary statistics are estimated via Monte Carlo. The objective of the current work is to improve on a variational implementation of the Bayesian synthetic likelihood introduced recently in the literature, to enable the application of that approach to high-dimensional problems. Here high-dimensional can mean problems with more than one hundred parameters. The improvements introduced relate to shrinkage estimation of covariance matrices in estimation of the synthetic likelihood, improved implementation of control variate approaches to stochastic gradient variance reduction, and parsimonious but expressive parametrizations of variational normal posterior covariance matrices in terms of factor structures to reduce the dimension of the optimization problem. The shrinkage covariance estimation is particularly important for stability of stochastic gradient optimization with noisy likelihood estimates. However, as the dimension increases, the quality of the posterior approximation deteriorates unless the number of Monte Carlo samples used to estimate the synthetic likelihood also increases. We explore the properties of the method in some real examples in cases where either the number of summary statistics, the number of model parameters, or both, are large.

Efficient likelihood computations for some multivariate Gaussian Markov random fields

Data collected from spatial locations are often multivariate. Gaussian conditional autoregressive (CAR) models, also known as Gaussian Markov random fields, are frequently used to analyze such continuous data, or as models for the parameters of discrete distributions. Two difficulties in Gaussian maximum likelihood estimation are ensuring that the parameter estimates are allowable values, and computing the likelihood efficiently. It is shown here that, for some commonly-used multivariate CAR models, checking for allowable parameter values can be facilitated, and the likelihood can be computed very quickly.