Joint Bayesian Estimation Research Articles

Joint Bayesian Channel Estimation and Data Detection for Underwater Acoustic Communications

Joint Bayesian Channel Estimation and Data Detection for Underwater Acoustic Communications

Joint Bayesian estimation of process and measurement noise statistics in nonlinear Kalman filtering

Bayesian filtering aims at sequentially estimating the states and parameters in nonlinear dynamical systems, and finds applications in a variety of engineering fields. Extended and unscented Kalman filter are popular choices for this purpose as they typically achieve a good trade-off between computational complexity and accuracy, while still quantifying posterior uncertainty. However, the accuracy of the identified states and parameters, both in terms of their posterior mean and variance, is heavily influenced by the assumed statistics of both the process and measurement noise terms. Therefore, learning the noise characteristics becomes an essential aspect of Bayesian filtering, but is non-trivial especially in cases where process and measurement noise statistics are jointly non-identifiable. This article proposes a methodology that decomposes measurement and process noise learning in a fully Bayesian setting. For measurement noise estimation, we improve upon a Bayesian method that leverages conjugacy of the inverse-Wishart distribution with respect to the Gaussian likelihood model. For process noise estimation, Bayesian model averaging is coupled with a mutation scheme to compare the performance of competing models while exploring high-probability regions in the space of the unknown process noise variance. The algorithm is first tested on simple numerical models for both state estimation and parameter learning, highlighting the superior accuracy of the proposed measurement noise correction and the benefits of decoupling process and measurement noise learning in non-identifiable settings. Then we demonstrate the usefulness of this algorithm in more challenging problems, namely identification of a multi-degree-of-freedom system under unmeasured excitation, and modeling of the highly nonlinear response of a full-scale bridge pier using experimental data.

Joint Bayesian estimation of cell dependence and gene associations in spatially resolved transcriptomic data

Recent technologies such as spatial transcriptomics, enable the measurement of gene expressions at the single-cell level along with the spatial locations of these cells in the tissue. Spatial clustering of the cells provides valuable insights into the understanding of the functional organization of the tissue. However, most such clustering methods involve some dimension reduction that leads to a loss of the inherent dependency structure among genes at any spatial location in the tissue. This destroys valuable insights of gene co-expression patterns apart from possibly impacting spatial clustering performance. In spatial transcriptomics, the matrix-variate gene expression data, along with spatial coordinates of the single cells, provides information on both gene expression dependencies and cell spatial dependencies through its row and column covariances. In this work, we propose a joint Bayesian approach to simultaneously estimate these gene and spatial cell correlations. These estimates provide data summaries for downstream analyses. We illustrate our method with simulations and analysis of several real spatial transcriptomic datasets. Our work elucidates gene co-expression networks as well as clear spatial clustering patterns of the cells. Furthermore, our analysis reveals that downstream spatial-differential analysis may aid in the discovery of unknown cell types from known marker genes.

Joint Bayesian Channel Estimation and Data Detection for OTFS Systems in LEO Satellite Communications

Lower earth orbit (LEO) satellites play an important role in the integration of space and terrestrial communication networks, which typically encounter high-mobility scenarios. It has been shown that orthogonal time frequency space (OTFS) modulation performs well in such high-mobility scenarios by transforming the time-varying channels into the delay-Doppler domain. In this paper, we develop a joint channel estimation and data detection algorithm for OTFS-based LEO satellite communications. Firstly, we adopt the powerful variational Bayesian inference (VBI) method for estimating the delay-Doppler channel vector, which contains the channel gain, the delay and the Doppler. Secondly, we exploit the unknown data symbols in an OTFS frame as ‘virtual pilots’ for improving the accuracy of channel estimation and detect them simultaneously. Our simulation results demonstrate that the proposed algorithm achieves improved channel estimation mean square error and bit error rate performance than its conventional counterparts.

Marginal unbiased score expansion and application to CMB lensing

We present the marginal unbiased score expansion (MUSE) method, an algorithm for generic high-dimensional hierarchical Bayesian inference. MUSE performs approximate marginalization over arbitrary non-Gaussian latent parameter spaces, yielding Gaussianized asymptotically unbiased and near-optimal constraints on global parameters of interest. It is computationally much cheaper than exact alternatives like Hamiltonian Monte Carlo (HMC), excelling on funnel problems which challenge HMC, and does not require any problem-specific user supervision like other approximate methods such as variational inference or many simulation-based inference methods. MUSE makes possible the first joint Bayesian estimation of the delensed Cosmic Microwave Background (CMB) power spectrum and gravitational lensing potential power spectrum, demonstrated here on a simulated data set as large as the upcoming South Pole Telescope 3G $1500\text{ }\text{ }{\mathrm{deg}}^{2}$ survey, corresponding to a latent dimensionality of $\ensuremath{\sim}6\text{ }\text{ }\mathrm{million}$ and of order 100 global bandpower parameters. On a subset of the problem where an exact but more expensive HMC solution is feasible, we verify that MUSE yields nearly optimal results. We also demonstrate that existing spectrum-based forecasting tools which ignore pixel-masking underestimate predicted error bars by only $\ensuremath{\sim}10%$. This method is a promising path forward for fast lensing and delensing analyses which will be necessary for future CMB experiments such as SPT-3G, Simons Observatory, or CMB-S4, and can complement or supersede existing HMC approaches. The success of MUSE on this challenging problem strengthens its case as a generic procedure for a broad class of high-dimensional inference problems.

Bayesian Estimation of Log-Normal Distribution Under Ranked Set Sampling With Missing Data

In this paper, joint Bayesian estimation of two parameters of a log-normal distribution is obtained based on simple random sampling (SRS) and ranked set sampling (RSS) with complete and missing data. For the missing data case, two missing mechanisms are considered: missing at random (MAR) and missing not at random (MNAR). A logistic regression model is specified for the MNAR mechanism. The results based on SRS and RSS with three different types of prior information are compared via simulation studies with both complete and missing data. It is shown that the results obtained under RSS are significantly better than those obtained under SRS. In the simulation studies, the Gibbs sampling method, Metropolis-Hastings algorithm and importance sampling method are used to sample posterior distributions to estimate the unknown parameters of the log-normal distribution.

Estimating effective population size changes from preferentially sampled genetic sequences.

Coalescent theory combined with statistical modeling allows us to estimate effective population size fluctuations from molecular sequences of individuals sampled from a population of interest. When sequences are sampled serially through time and the distribution of the sampling times depends on the effective population size, explicit statistical modeling of sampling times improves population size estimation. Previous work assumed that the genealogy relating sampled sequences is known and modeled sampling times as an inhomogeneous Poisson process with log-intensity equal to a linear function of the log-transformed effective population size. We improve this approach in two ways. First, we extend the method to allow for joint Bayesian estimation of the genealogy, effective population size trajectory, and other model parameters. Next, we improve the sampling time model by incorporating additional sources of information in the form of time-varying covariates. We validate our new modeling framework using a simulation study and apply our new methodology to analyses of population dynamics of seasonal influenza and to the recent Ebola virus outbreak in West Africa.

Joint Bayesian Estimation of Voxel Activation and Inter-regional Connectivity in fMRI Experiments.

Brain activation and connectivity analyses in task-based functional magnetic resonance imaging (fMRI) experiments with multiple subjects are currently at the forefront of data-driven neuroscience. In such experiments, interest often lies in understanding activation of brain voxels due to external stimuli and strong association or connectivity between the measurements on a set of pre-specified groups of brain voxels, also known as regions of interest (ROI). This article proposes a joint Bayesian additive mixed modeling framework that simultaneously assesses brain activation and connectivity patterns from multiple subjects. In particular, fMRI measurements from each individual obtained in the form of a multi-dimensional array/tensor at each time are regressed on functions of the stimuli. We impose a low-rank parallel factorization decomposition on the tensor regression coefficients corresponding to the stimuli to achieve parsimony. Multiway stick-breaking shrinkage priors are employed to infer activation patterns and associated uncertainties in each voxel. Further, the model introduces region-specific random effects which are jointly modeled with a Bayesian Gaussian graphical prior to account for the connectivity among pairs of ROIs. Empirical investigations under various simulation studies demonstrate the effectiveness of the method as a tool to simultaneously assess brain activation and connectivity. The method is then applied to a multi-subject fMRI dataset from a balloon-analog risk-taking experiment, showing the effectiveness of the model in providing interpretable joint inference on voxel-level activations and inter-regional connectivity associated with how the brain processes risk. The proposed method is also validated through simulation studies and comparisons to other methods used within the neuroscience community.

Inefficiency and bank failure: A joint Bayesian estimation method of stochastic frontier and hazards models

We propose a Bayesian one-stage approach to estimate the effect of inefficiency on the time to failure (bankruptcy) of U.S. commercial banks. We do so combining stochastic frontier and proportional hazards settings. Most of the existing literature use two-stage methods which may yield inefficient, biased, and inconsistent estimates. Our proposal overcomes these issues, allows computing the marginal distribution of inefficiencies for each observational unit, and facilitates statistical inference of non-linear functions of parameters such as returns to scale. Simulation exercises show that our proposal outperforms the two-stage maximum likelihood approach traditionally used in the literature. In addition, empirical evidence suggests that inefficiency of U.S. commercial banks during the global financial crisis in 2008–2009 played a statistically and economically significant role determining the time to failure.

Information-Based Search for an Atmospheric Release Using a Mobile Robot: Algorithm and Experiments

Finding the location and strength of an unknown hazardous release is of paramount importance in emergency response and environmental monitoring; thus, it has been an active research area for several years known as source term estimation (STE). This paper presents a joint Bayesian estimation and planning algorithm to guide a mobile robot to collect informative measurements, allowing the source parameters to be estimated quickly and accurately. The estimation is performed recursively using Bayes’ theorem, where uncertainties in the meteorological and dispersion parameters are considered and the intermittent readings from a low-cost gas sensor are addressed by a novel likelihood function. The planning strategy is designed to maximize the expected utility function based on the estimated information gain of the source parameters. Subsequently, this paper presents the first experimental result of such a system in turbulent, diffusive conditions, in which a ground robot equipped with the low-cost gas sensor responds to the hazardous source simulated by incense sticks. The experimental results demonstrate the effectiveness of the proposed estimation and search algorithm for STE based on the mobile robot and the low-cost sensor.

Joint Bayesian estimation of tensor and lensing B modes in the power spectrum of CMB polarization data

We investigate the performance of a simple Bayesian fitting approach to correct the cosmic microwave background (CMB) B-mode polarization for gravitational lensing effects in the recovered probability distribution of the tensor-to-scalar ratio. We perform a two-dimensional power spectrum fit of the amplitude of the primordial B-modes (tensor-to-scalar ratio, $r$) and the amplitude of the lensing B-modes (parameter $A_{lens}$), jointly with the estimation of the astrophysical foregrounds including both synchrotron and thermal dust emissions. Using this Bayesian framework, we forecast the ability of the proposed CMB space mission LiteBIRD to constrain $r$ in the presence of realistic lensing and foreground contributions. We compute the joint posterior distribution of $r$ and $A_{lens}$, which we improve by adopting a prior on $A_{lens}$ taken from the South Pole Telescope (SPT) measurement. As it applies to the power spectrum, this approach cannot mitigate the uncertainty on $r$ that is due to E-mode cosmic variance transferred to B-modes by lensing, unlike standard delensing techniques that are performed on maps. However, the method allows to correct for the bias on $r$ induced by lensing, at the expense of a larger uncertainty due to the increased volume of the parameter space. We quantify, for different values of the tensor-to-scalar ratio, the trade-off between bias correction and increase of uncertainty on $r$. For LiteBIRD simulations, which include foregrounds and lensing contamination, we find that correcting the foreground-cleaned CMB B-mode power spectrum for the lensing bias, not the lensing cosmic variance, still guarantees a $3\sigma$ detection of $r=5\times 10^{-3}$. The significance of the detection is increased to $6\sigma$ when the current SPT prior on $A_{lens}$ is adopted.

Joint Bayesian Estimation of Time-Varying LP Parameters and Excitation for Speech

We consider the joint estimation of time-varying linear prediction (TVLP) filter coefficients and the excitation signal parameters for the analysis of long-term speech segments. Traditional approaches to TVLP estimation assume linear expansion of the coefficients in a set of known basis functions only. But, excitation signal is also time-varying, which affects the estimation of TVLP filter parameters. In this letter, we propose a Bayesian approach, to incorporate the nature of excitation signal and also adapt regularization of the filter parameters. Since the order of the system is not known a priori, we formulate a Gaussian prior for the filter parameters, and the excitation signal is modeled as Gaussian with time-varying Gamma distributed precision. We develop an iterative algorithm for the maximum-likelihood estimation of the posterior distribution of filter parameters and the time-varying precision of the excitation signal, along with the parameters of the prior distribution. We show that the proposed method adapts to different types of excitation signals in speech, and also the time-varying system with unknown model order. The spectral modeling performance for synthetic speech-like signals, quantified using the absolute spectral difference shows that the proposed method estimates the system function more accurately compared to several of the traditional methods.

A Message-Passing Receiver for OFDM-Based Self-Interference-Limited Networks

In this paper, we develop a reliable bandwidth-efficient solution for coherent reception in networks prone to self-interference. Our primary focus is on orthogonal-frequency-division-multiplexing (OFDM)-based amplify-and-forward (AF) two-way relaying, where each source node suffers from reradiation of its own signal. An important issue is the superposition of the desired and interfering signals, which makes the channel estimation task difficult, particularly for varying channel coefficients. To address this challenging problem, we introduce a factor graph model of the estimation and detection problem at hand. Exploiting the conditional independence between unknown variables in the factor graph, an efficient message-passing algorithm, which performs joint Bayesian channel estimation, self-interference mitigation, and decoding, is derived. Simulation results show that the performance of the proposed method is close to that given by perfect knowledge of the channel for the target and interfering signals, even with limited training overhead.

Joint Bayesian wavelet and well-path estimation in the impedance domain

We addressed the problem of the well-to-seismic tie as a Bayesian inversion for the wavelet and well path in the impedance domain. The result of the joint inversion is a set of wavelets for multiple angle stacks, and a corresponding well path. The wavelet optimally links the impedance data along the well to the seismic data along the optimized well path in the seismic time domain. Starting with prior distribution for wavelet and well path, the method calculates the posterior distribution of conditioning the prior distributions with the seismic and well-log data. This is done by iteratively inverting the seismic data with the current wavelet, to obtain an impedance cube around the well. In a second step, the seismic impedances are projected onto the well path. By minimizing the misfit between the inverted seismic impedances and the impedances derived from the well log, the wavelet and well path are optimized. Comparing the well and seismic data in the impedance domain enables the method to work on short and noisy well logs. Another advantage of this method is its ability to derive wavelets for multiple angle stacks and multiple well locations simultaneously. We tested the method on synthetic and real data examples. The algorithm performed well in the synthetic examples, in which we had control over the modeling wavelet, and the wavelets derived for a real data example showed consistently good seismic-to-well ties for six angle stacks and seven wells. The main algorithm we developed was aimed to linearize the problem. We compared the posterior distribution of the linearized result with a sampling-based result in a real data example and found good agreement.

Joint Bayesian Estimation of Close Subspaces from Noisy Measurements

In this letter, we consider two sets of observations defined as subspace signals embedded in noise and we wish to analyze the distance between these two subspaces. The latter entails evaluating the angles between the subspaces, an issue reminiscent of the well-known Procrustes problem. A Bayesian approach is investigated where the subspaces of interest are considered as random with a joint prior distribution (namely a Bingham distribution), which allows the closeness of the two subspaces to be adjusted. Within this framework, the minimum mean-square distance estimator of both subspaces is formulated and implemented via a Gibbs sampler. A simpler scheme based on alternative maximum a posteriori estimation is also presented. The new schemes are shown to provide more accurate estimates of the angles between the subspaces, compared to singular value decomposition based independent estimation of the two subspaces.

Parameter estimation of multi-component chirp signals based on discrete chirp Fourier transform and population Monte Carlo

In this paper, a parameter estimation method for multi-component chirp signals with white Gaussian noise is proposed based on the modified discrete chirp Fourier transform (MDCFT) and the population Monte Carlo (PMC) methodology, in which the model order is unknown. By utilizing the integrability of linear parameters in the Bayesian model, this paper considers the posterior distribution of nonlinear parameters. MDCFT was adopted to calculate the chirpogram of the observed data, and clear peaks can be detected in the discrete chirp Fourier transform domain. The importance function (IF) was constructed according to the peaks, and the PMC algorithm was employed to evaluate the posterior distribution. The proposed method cannot only use the selected IF to generate the sample fitting target function in the parameter region of interest, but can also utilize samples and importance weights to update the IF adaptively. The simulation results indicated that the proposed method can realize joint Bayesian model selection and parameter estimation of multi-component chirp signals. Compared with the two existing methods based on Monte Carlo methodology, the proposed method exhibits improved performance.

Sound Source Localization Using Joint Bayesian Estimation With a Hierarchical Noise Model

The performance of sound source localization is often reduced by the presence of colored noise in the environment, such as room reverberation. In this study, a method for estimating the noise spatial covariance using a hierarchical model is proposed and its performance is evaluated. By employing the hierarchical model in joint Bayesian estimation, robust estimation of the covariance is expected with a relatively small amount of data. Moreover, a method of jointly estimating the number of sources is introduced so that it can be used for cases in which the number of active sources dynamically changes, for example, speech signals. The results of the experiments performed using actual room reverberation show the effectiveness of the proposed method.

Comments on “Joint Bayesian Model Selection and Estimation of Noisy Sinusoids Via Reversible Jump MCMC”

Reversible jump MCMC (RJ-MCMC) sampling techniques, which allow to jointly tackle model selection and parameter estimation problems in a coherent Bayesian framework, have become increasingly popular in the signal processing literature since the seminal paper of Andrieu and Doucet [“Joint Bayesian model selection and estimation of noisy sinusoids via reversible jump MCMC,” IEEE Trans. Signal Process, vol. 47, no. 10, pp. 2667-2676, 1999]. Crucial to the implementation of any RJ-MCMC sampler is the computation of the so-called Metropolis-Hastings-Green (MHG) ratio, which determines the acceptance probability for the proposed moves. It turns out that the expression of the MHG ratio that was given in the paper of Andrieu and Doucet for “Birth-or-Death” moves is erroneous and has been reproduced in many subsequent papers dealing with RJ-MCMC sampling in the signal processing literature. This note fixes the erroneous expression and briefly discusses its cause and consequences.

Bayesian Equalization for LDPC Channel Decoding

We describe the channel equalization problem, and its prior estimate of the channel state information (CSI), as a joint Bayesian estimation problem to improve each symbol posterior estimates at the input of the channel decoder. Our approach takes into consideration not only the uncertainty due to the noise in the channel, but also the uncertainty in the CSI estimate. However, this solution cannot be computed in linear time, because it depends on all the transmitted symbols. Hence, we also put forward an approximation for each symbol's posterior, using the expectation propagation algorithm, which is optimal from the Kullback-Leibler divergence viewpoint and yields an equalization with a complexity identical to the BCJR algorithm. We also use a graphical model representation of the full posterior, in which the proposed approximation can be readily understood. The proposed posterior estimates are more accurate than those computed using the ML estimate for the CSI. In order to illustrate this point, we measure the error rate at the output of a low-density parity-check decoder, which needs the exact posterior for each symbol to detect the incoming word and it is sensitive to a mismatch in those posterior estimates. For example, for QPSK modulation and a channel with three taps, we can expect gains over 0.5 dB with same computational complexity as the ML receiver.

Joint NDT Image Restoration and Segmentation Using Gauss–Markov–Potts Prior Models and Variational Bayesian Computation

In this paper, we propose a method to simultaneously restore and to segment piecewise homogeneous images degraded by a known point spread function (PSF) and additive noise. For this purpose, we propose a family of nonhomogeneous Gauss-Markov fields with Potts region labels model for images to be used in a Bayesian estimation framework. The joint posterior law of all the unknowns (the unknown image, its segmentation (hidden variable) and all the hyperparameters) is approximated by a separable probability law via the variational Bayes technique. This approximation gives the possibility to obtain practically implemented joint restoration and segmentation algorithm. We will present some preliminary results and comparison with a MCMC Gibbs sampling based algorithm. We may note that the prior models proposed in this work are particularly appropriate for the images of the scenes or objects that are composed of a finite set of homogeneous materials. This is the case of many images obtained in nondestructive testing (NDT) applications.