Joint Maximum A Posteriori Research Articles

Dynamical system identification, model selection, and model uncertainty quantification by Bayesian inference.

This study presents a Bayesian maximum a posteriori (MAP) framework for dynamical system identification from time-series data. This is shown to be equivalent to a generalized Tikhonov regularization, providing a rational justification for the choice of the residual and regularization terms, respectively, from the negative logarithms of the likelihood and prior distributions. In addition to the estimation of model coefficients, the Bayesian interpretation gives access to the full apparatus for Bayesian inference, including the ranking of models, the quantification of model uncertainties, and the estimation of unknown (nuisance) hyperparameters. Two Bayesian algorithms, joint MAP and variational Bayesian approximation, are compared to the least absolute shrinkage and selection operator (LASSO), ridge regression, and the sparse identification of nonlinear dynamics (SINDy) algorithms for sparse regression by application to several dynamical systems with added Gaussian or Laplace noise. For multivariate Gaussian likelihood and prior distributions, the Bayesian formulation gives Gaussian posterior and evidence distributions, in which the numerator terms can be expressed in terms of the Mahalanobis distance or "Gaussian norm" ||y-y^||M-12=(y-y^)⊤M-1(y-y^), where y is a vector variable, y^ is its estimator, and M is the covariance matrix. The posterior Gaussian norm is shown to provide a robust metric for quantitative model selection for the different systems and noise models examined.

A Unified Bayesian Framework for Joint Estimation and Anomaly Detection in Environmental Sensor Networks

Advanced large-scale environmental monitoring systems relying on the emerging aerial/terrestrial technologies of wireless sensor networks (WSNs), unmanned aerial vehicles (UAVs), and mobile crowdsensing, impose strong requirements on the reliability of the collected data. Unfortunately, sensing units can suddenly suffer unexpected anomalies due to accidental faults or malicious causes. Outlier detection methods have been widely employed to identify and discard unreliable measurements from large data sets, but further improvements in the sensing processes can be obtained by adopting advanced signal processing algorithms that take full advantage of all the collected information without rejecting the measurements. In this paper, we propose a novel unified Bayesian framework that enable simultaneous estimation of a common parameter of interest and identification of multiple and possibly different types of anomalies that can affect sensors in environmental sensor networks. Specifically, we consider two rather general error models based on Gaussian mixtures able to capture different variations affecting the quality of the collected measurements. For each model, we illustrate the optimal joint maximum-likelihood and maximum a-posteriori (ML-MAP) estimation method, which represents the benchmark for the problem at hand, and propose novel reduced-complexity two-step algorithms able to achieve almost the same performance of the joint ML-MAP, but at a fraction of its computational cost. The derivations of all the algorithms are also extended to handle the more general case in which the probability of occurrence of anomalies is unknown and should be inferred from the data using an Empirical Bayes approach. Extensive performance analyses using both synthetic and real experimental data acquired in a network of environmental monitoring stations deployed in the Apulia region, south of Italy, demonstrate the effectiveness of the proposed framework.

Joint MAP channel estimation and data detection for OFDM in presence of phase noise from free running and phase locked loop oscillator

This paper addresses a computationally compact and statistically optimal joint Maximum a Posteriori (MAP) algorithm for channel estimation and data detection in the presence of Phase Noise (PHN) in iterative Orthogonal Frequency Division Multiplexing (OFDM) receivers used for high speed and high spectral efficient wireless communication systems. The MAP cost function for joint estimation and detection is derived and optimized further with the proposed cyclic gradient descent optimization algorithm. The proposed joint estimation and detection algorithm relaxes the restriction of small PHN assumptions and utilizes the prior statistical knowledge of PHN spectral components to produce a statistically optimal solution. The frequency-domain estimation of Channel Transfer Function (CTF) in frequency selective fading makes the method simpler, compared with the estimation of Channel Impulse Response (CIR) in the time domain. Two different time-varying PHN models, produced by Free Running Oscillator (FRO) and Phase-Locked Loop (PLL) oscillator, are presented and compared for performance difference with proposed OFDM receiver. Simulation results for joint MAP channel estimation are compared with Cramer-Rao Lower Bound (CRLB), and the simulation results for joint MAP data detection are compared with “NO PHN” performance to demonstrate that the proposed joint MAP estimation and detection algorithm achieve near-optimum performance even under multipath channel fading.

Joint MAP Detection for OFDM in Presence of Phase Noise from Free Running and Phase Locked Loop Oscillator

This paper addresses a statistically optimal joint maximum a posteriori (MAP) data detection and phase noise (PHN) estimation algorithm for iterative orthogonal frequency division multiplexing (OFDM) receivers, used for high speed and high spectral efficient wireless communication systems. First the cost function for data detection in presence of PHN is formulated with MAP criterion and is optimized further. The optimization is performed in frequency domain with proposed multidimensional circulatory gradient descent algorithm. With reduced computational complexity, the proposed optimization incorporates the statistics of PHN spectral components in prior. The condition of small PHN is relaxed in the proposed algorithm to achieve near optimal performance. Two different models for time varying PHN: produced by free running oscillator and phase locked loop (PLL) oscillator, are presented and implemented for proposed OFDM receiver. Simulation results demonstrate that the proposed optimization achieves “NO PHN” performance for both the PHN models even under multipath channel fading.

On variational solutions for whole brain serial-section histology using a Sobolev prior in the computational anatomy random orbit model.

This paper presents a variational framework for dense diffeomorphic atlas-mapping onto high-throughput histology stacks at the 20 μm meso-scale. The observed sections are modelled as Gaussian random fields conditioned on a sequence of unknown section by section rigid motions and unknown diffeomorphic transformation of a three-dimensional atlas. To regularize over the high-dimensionality of our parameter space (which is a product space of the rigid motion dimensions and the diffeomorphism dimensions), the histology stacks are modelled as arising from a first order Sobolev space smoothness prior. We show that the joint maximum a-posteriori, penalized-likelihood estimator of our high dimensional parameter space emerges as a joint optimization interleaving rigid motion estimation for histology restacking and large deformation diffeomorphic metric mapping to atlas coordinates. We show that joint optimization in this parameter space solves the classical curvature non-identifiability of the histology stacking problem. The algorithms are demonstrated on a collection of whole-brain histological image stacks from the Mouse Brain Architecture Project.

Cyclic gradient descent optimisation for joint MAP estimation of channel and phase noise in orthogonal frequency division multiplexing

This study addresses the statistically optimal channel estimation for orthogonal frequency division multiplexing systems in the presence of phase noise (PHN). The maximum a posteriori (MAP) cost function for the joint estimation of channel transfer function (CTF) and PHN is presented, which utilises the prior statistical knowledge of PHN spectral components without the assumption of small PHN. The frequency-domain estimation of unknown frequency selective fading makes the method simpler, compared with the estimation of channel impulse response in time domain. An iterative cyclic gradient descent optimisation algorithm is derived to minimise the cost function over the unknown CTF and PHN. Cramer-Rao lower bound is analysed and compared with the simulation results to illustrate that the proposed joint MAP estimation can achieve near optimum performance.

Research on adaptive optics image restoration algorithm based on improved joint maximum a posteriori method

In this paper, we propose a point spread function (PSF) reconstruction method and joint maximum a posteriori (JMAP) estimation method for the adaptive optics image restoration. Using the JMAP method as the basic principle, we establish the joint log likelihood function of multi-frame adaptive optics (AO) images based on the image Gaussian noise models. To begin with, combining the observed conditions and AO system characteristics, a predicted PSF model for the wavefront phase effect is developed; then, we build up iterative solution formulas of the AO image based on our proposed algorithm, addressing the implementation process of multi-frame AO images joint deconvolution method. We conduct a series of experiments on simulated and real degraded AO images to evaluate our proposed algorithm. Compared with the Wiener iterative blind deconvolution (Wiener-IBD) algorithm and Richardson-Lucy IBD algorithm, our algorithm has better restoration effects including higher peak signal-to-noise ratio (PSNR) and Laplacian sum (LS) value than the others. The research results have a certain application values for actual AO image restoration.

Estimation of channel and Carrier Frequency Offset in OFDM systems using joint statistical framework

Joint estimation of channel and Carrier Frequency Offset (CFO) in Orthogonal Frequency Division Multiplexing (OFDM) systems, using a Statistical framework, is shown in this paper. Hybrid Cramér–RaoLower Bounds (HCRLBs) for the estimation of CFO together with the channel are obtained. The significance of prior information in the formulation of a joint estimator is shown by comparing HCRLB with the corresponding standard CRLB. We propose a Joint Maximum a posteriori (JMAP) algorithm for the estimation of channel and CFO in OFDM, utilizing the prior statistical knowledge of channel. To reduce the complexity of JMAP estimator, a Modified JMAP (MJMAP) algorithm, which has no grid searches, is also proposed. The estimation methods are analyzed by numerical simulations and resultant conclusions validate the better performance of the proposed algorithms when compared with previous algorithms.

Joint statistical framework for the estimation of channel and SFO in OFDM systems

Joint estimation of channel and sampling frequency offset (SFO) in orthogonal frequency division multiplexing (OFDM) systems, using Bayesian framework, is shown in this study. Hybrid Cramér–Rao lower bounds (HCRLBs) for the estimation of SFO together with channel are obtained. The significance of Bayesian approach in the formulation of joint estimator is shown by comparing HCRLB with the corresponding standard CRLB. The authors propose a joint maximum a posteriori (JMAP) algorithm for the estimation of channel and SFO in OFDM, utilising the prior statistical knowledge of channel. To reduce the complexity of JMAP estimator, a modified MAP algorithm, which has no grid searches, is also proposed. Also, they analyse the effect of inaccurate knowledge of channel statistics and signal-to-noise ratio on the estimation accuracy. The estimation methods are analysed by numerical simulations and resultant conclusions validate the better performance of the proposed algorithms when compared with previous algorithms.

Adaptive Spatial Regularization Sparse Unmixing Strategy Based on Joint MAP for Hyperspectral Remote Sensing Imagery

Sparse unmixing, as a recently developed spectral unmixing approach, has been successfully applied based on the assumption that the observed image signatures can be expressed in an efficient linear sparse regression with the potentially very large endmember spectral library. To improve the unmixing accuracy, spatial information has been incorporated in the sparse unmixing formulation by adding an appropriate spatial regularization for the hyperspectral remote sensing imagery. However, for the traditional spatial regularization sparse unmixing (SRSU) algorithms, it is a difficult task to set appropriate user-defined regularization parameters in real applications, and this often has a high computational cost. To overcome the difficulty of the regularization parameter selection, the adaptive spatial regularization sparse unmixing (ASRSU) strategy based on the joint maximum a posteriori (JMAP) estimation technique is proposed in this paper. In ASRSU, the SRSU problem is formulated in the framework of JMAP with an appropriate prior model. ASRSU considers the regularization parameters and the abundances jointly by an alternating iterative process, and the relationships between the regularization parameters and the abundances are obtained from the JMAP model. Based on the ASRSU strategy, two ASRSU algorithms are presented: the adaptive total variation spatial regularization sparse unmixing algorithm and the adaptive nonlocal means filtering sparse unmixing algorithm. The experimental results demonstrate that the two proposed ASRSU algorithms based on JMAP can adaptively obtain optimal or near-optimal regularization parameters for the three simulated datasets and the two real hyperspectral remote sensing images.

Iterative joint MAP single-channel speech enhancement given non-uniform phase prior

Within the last three decades research in single-channel speech enhancement has been mainly focused on filtering the noisy spectral amplitude without that much focus on the integration of phase-based signal processing. Recently, several phase-aware algorithms based on phase-sensitive signal models were proposed for speech enhancement using the minimum mean squared error (MMSE). Improved performance over the conventional phase-insensitive approaches has been achieved. In this paper, we propose an iterative joint maximum a posteriori (MAP) amplitude and phase estimator (ijMAP) assuming a non-uniform phase distribution. Experimental results demonstrate the effectiveness of the proposed method in recovering both amplitude and phase in noise, justified by perceived quality, speech intelligibility and phase estimation error instrumental measures. The proposed method, brings joint improvement in perceived quality and speech intelligibility compared to the phase-blind joint MAP estimator with a comparable performance to the complex MMSE estimator.

Precise periodic components estimation for chronobiological signals through Bayesian Inference with sparsity enforcing prior.

The toxicity and efficacy of more than 30 anticancer agents present very high variations, depending on the dosing time. Therefore, the biologists studying the circadian rhythm require a very precise method for estimating the periodic component (PC) vector of chronobiological signals. Moreover, in recent developments, not only the dominant period or the PC vector present a crucial interest but also their stability or variability. In cancer treatment experiments, the recorded signals corresponding to different phases of treatment are short, from 7 days for the synchronization segment to 2 or 3 days for the after-treatment segment. When studying the stability of the dominant period, we have to consider very short length signals relative to the prior knowledge of the dominant period, placed in the circadian domain. The classical approaches, based on Fourier transform (FT) methods are inefficient (i.e., lack of precision) considering the particularities of the data (i.e., the short length). Another particularity of the signals considered in such experiments is the level of noise: such signals are very noisy and establishing the periodic components that are associated with the biological phenomena and distinguishing them from the ones associated with the noise are difficult tasks. In this paper, we propose a new method for the estimation of the PC vector of biomedical signals, using the biological prior informations and considering a model that accounts for the noise. The experiments developed in cancer treatment context are recording signals expressing a limited number of periods. This is a prior information that can be translated as the sparsity of the PC vector. The proposed method considers the PC vector estimation as an Inverse Problem (IP) using the general Bayesian inference in order to infer the unknown of our model, i.e. the PC vector but also the hyperparameters (i.e the variances). The sparsity prior information is modeled using a sparsity enforcing prior law. In this paper, we propose a Student’s t distribution, viewed as the marginal distribution of a bivariate normal-inverse gamma distribution. We build a general infinite Gaussian scale mixture (IGSM) hierarchical model where we assign prior distributions also for the hyperparameters. The expression of the joint posterior law of the unknown PC vector and hyperparameters is obtained via Bayes rule, and then, the unknowns are estimated via joint maximum a posteriori (JMAP) or posterior mean (PM). For the PM estimator, the expression of the posterior distribution is approximated by a separable one, via variational Bayesian approximation (VBA), using the Kullback-Leibler (KL) divergence. For the PM estimation, two possibilities are considered: an approximation with a partially separable distribution and an approximation with a fully separable one. Both resulting algorithms corresponding to the PM estimation and the one corresponding to the JMAP estimation are iterative algorithms. The algorithms are presented in detail and are compared with the ones corresponding to the Gaussian model. We examine the convergency of the algorithms and give simulation results to compare their performances. Finally, we show simulation results on synthetic and real data in cancer treatment applications. The real data considered in this paper examines the rest-activity patterns of KI/KI Per2::luc mouse, aged 10 weeks, singly housed in RealTime Biolumicorder (RT-BIO).Electronic supplementary materialThe online version of this article (doi:10.1186/s13637-015-0033-6) contains supplementary material, which is available to authorized users.

Optimal Joint Decoding of Correlated Data Over Orthogonal Multiple Access Channels with Memory

Motivated by the structure of basic sensor networks, we study an optimal joint decoding problem in which the real-valued outputs of two correlated Gaussian sources are scalar quantized, bit assigned, and transmitted, without applying channel coding or interleaving, over a multiple-access channel that consists of two orthogonal point-to-point time-correlated Rayleigh fading subchannels used with soft-decision demodulation. Each fading subchannel is modeled by a nonbinary Markov noise discrete channel that was recently shown to effectively represent it. The correlated sources have memory captured by a time-varying correlation coefficient governed by a two-state first-order Markov process. At the receiver side, we design a joint sequence maximum a posteriori (MAP) decoder to exploit the correlation between the two sources, their temporal memory, and the redundancy left in the quantizers' indexes, the channels' soft-decision outputs, and noise memory. Under the simple practical case of using two-level source quantization, we propose a Markov model to estimate the joint behavior of the quantized sources. We then establish necessary and sufficient conditions under which the delay-prone joint sequence MAP decoder can be reduced to a simple instantaneous symbol-by-symbol decoder. We illustrate our analytical results by system simulation and demonstrate that joint MAP decoding can appropriately harness source and channel characteristics to achieve improved signal-to-distortion ratio performance for a wide range of system conditions.

Bayesian sparse solutions to linear inverse problems with non-stationary noise with Student-t priors

Bayesian approach has become a commonly used method for inverse problems arising in signal and image processing. One of the main advantages of the Bayesian approach is the possibility to propose unsupervised methods where the likelihood and prior model parameters can be estimated jointly with the main unknowns. In this paper, we propose to consider linear inverse problems in which the noise may be non-stationary and where we are looking for a sparse solution. To consider both of these requirements, we propose to use Student-t prior model both for the noise of the forward model and the unknown signal or image. The main interest of the Student-t prior model is its Infinite Gaussian Scale Mixture (IGSM) property. Using the resulted hierarchical prior models we obtain a joint posterior probability distribution of the unknowns of interest (input signal or image) and their associated hidden variables. To be able to propose practical methods, we use either a Joint Maximum A Posteriori (JMAP) estimator or an appropriate Variational Bayesian Approximation (VBA) technique to compute the Posterior Mean (PM) values. The proposed method is applied in many inverse problems such as deconvolution, image restoration and computed tomography. In this paper, we show only some results in signal deconvolution and in periodic components estimation of some biological signals related to circadian clock dynamics for cancer studies.

Exploring Multimodal Data Fusion Through Joint Decompositions with Flexible Couplings

A Bayesian framework is proposed to define flexible coupling models for joint tensor decompositions of multiple data sets. Under this framework, a natural formulation of the data fusion problem is to cast it in terms of a joint maximum a posteriori (MAP) estimator. Data driven scenarios of joint posterior distributions are provided, including general Gaussian priors and non Gaussian coupling priors. We present and discuss implementation issues of algorithms used to obtain the joint MAP estimator. We also show how this framework can be adapted to tackle the problem of joint decompositions of large datasets. In the case of a conditional Gaussian coupling with a linear transformation, we give theoretical bounds on the data fusion performance using the Bayesian Cramer-Rao bound. Simulations are reported for hybrid coupling models ranging from simple additive Gaussian models, to Gamma-type models with positive variables and to the coupling of data sets which are inherently of different size due to different resolution of the measurement devices.

Adaptive non-local Euclidean medians sparse unmixing for hyperspectral imagery

Sparse unmixing models based on sparse representation theory and a sparse regression model have been successfully applied to hyperspectral remote sensing image unmixing. To better utilize the abundant spatial information and improve the unmixing accuracy, spatial sparse unmixing methods such as the non-local sparse unmixing (NLSU) approach have been proposed. Although the NLSU method utilizes non-local spatial information as the spatial regularization term and obtains a satisfactory unmixing accuracy, the final abundances are affected by the non-local neighborhoods and drift away from the true abundance values when the observed hyperspectral images have high noise levels. Furthermore, NLSU contains two regularization parameters which need to be appropriately set in real applications, which is a difficult task and often has a high computational cost. To solve these problems, an adaptive non-local Euclidean medians sparse unmixing (ANLEMSU) method is proposed to improve NLSU by replacing the non-local means total variation spatial consideration with the non-local Euclidean medians filtering approach. In addition, ANLEMSU utilizes a joint maximum a posteriori (JMAP) strategy to acquire the relationships between the regularization parameters and the estimated abundances, and achieves the fractional abundances adaptively, without the need to set the two regularization parameters manually. The experimental results using both simulated data and real hyperspectral images indicate that ANLEMSU outperforms the previous sparse unmixing algorithms and, hence, provides an effective option for the unmixing of hyperspectral remote sensing imagery.

A full‐spectral Bayesian reconstruction approach based on the material decomposition model applied in dual‐energy computed tomography

Dual-energy computed tomography (DECT) makes it possible to get two fractions of basis materials without segmentation. One is the soft-tissue equivalent water fraction and the other is the hard-matter equivalent bone fraction. Practical DECT measurements are usually obtained with polychromatic x-ray beams. Existing reconstruction approaches based on linear forward models without counting the beam polychromaticity fail to estimate the correct decomposition fractions and result in beam-hardening artifacts (BHA). The existing BHA correction approaches either need to refer to calibration measurements or suffer from the noise amplification caused by the negative-log preprocessing and the ill-conditioned water and bone separation problem. To overcome these problems, statistical DECT reconstruction approaches based on nonlinear forward models counting the beam polychromaticity show great potential for giving accurate fraction images. This work proposes a full-spectral Bayesian reconstruction approach which allows the reconstruction of high quality fraction images from ordinary polychromatic measurements. This approach is based on a Gaussian noise model with unknown variance assigned directly to the projections without taking negative-log. Referring to Bayesian inferences, the decomposition fractions and observation variance are estimated by using the joint maximum a posteriori (MAP) estimation method. Subject to an adaptive prior model assigned to the variance, the joint estimation problem is then simplified into a single estimation problem. It transforms the joint MAP estimation problem into a minimization problem with a nonquadratic cost function. To solve it, the use of a monotone conjugate gradient algorithm with suboptimal descent steps is proposed. The performance of the proposed approach is analyzed with both simulated and experimental data. The results show that the proposed Bayesian approach is robust to noise and materials. It is also necessary to have the accurate spectrum information about the source-detector system. When dealing with experimental data, the spectrum can be predicted by a Monte Carlo simulator. For the materials between water and bone, less than 5% separation errors are observed on the estimated decomposition fractions. The proposed approach is a statistical reconstruction approach based on a nonlinear forward model counting the full beam polychromaticity and applied directly to the projections without taking negative-log. Compared to the approaches based on linear forward models and the BHA correction approaches, it has advantages in noise robustness and reconstruction accuracy.

A Data-Driven Approach to A Priori SNR Estimation

The a priori signal-to-noise ratio (SNR) plays an important role in many speech enhancement algorithms. In this paper, we present a data-driven approach to a priori SNR estimation. It may be used with a wide range of speech enhancement techniques, such as, e.g., the minimum mean square error (MMSE) (log) spectral amplitude estimator, the super Gaussian joint maximum a posteriori (JMAP) estimator, or the Wiener filter. The proposed SNR estimator employs two trained artificial neural networks, one for speech presence, one for speech absence. The classical decision-directed a priori SNR estimator by Ephraim and Malah is broken down into its two additive components, which now represent the two input signals to the neural networks. Both output nodes are combined to represent the new a priori SNR estimate. As an alternative to the neural networks, also simple lookup tables are investigated. Employment of these data-driven nonlinear a priori SNR estimators reduces speech distortion, particularly in speech onset, while retaining a high level of noise attenuation in speech absence.

Estimation of the parameters in regularized simultaneous super-resolution

We describe a method for automatic determination of the regularization parameters for the class of simultaneous super-resolution (SR) algorithms. This method, proposed in ( Zibetti et al., 2008c), is based on the joint maximum a posteriori (JMAP) estimation technique, which is a fast alternative to estimate the parameters. However, the classical JMAP technique can be unstable and may generate multiple local minima. In order to stabilize the JMAP estimation, while achieving a cost function with a unique global solution, we derive an improved solution by modeling the JMAP hyperparameters with a gamma prior distribution. In this work, experimental results are provided to illustrate the effectiveness of the proposed method for automatic determination of the regularization parameters for the simultaneous SR. Moreover, we contrast the proposed method to a reference method with known fixed parameters as well as to other parameter selection methods based on the L-curve. These results validate the proposed method as a very attractive alternative for estimating the regularization parameters.

Determining the regularization parameters for super-resolution problems

We derive a novel method to determine the parameters for regularized super-resolution problems, addressing both the traditional regularized super-resolution problem with single- and multiple-parameters and the simultaneous super-resolution problem with two parameters. The proposal relies on the joint maximum a posteriori (JMAP) estimation technique. The classical JMAP technique provides solutions at low computational cost, but it may be unstable and presents multiple local minima. We propose to stabilize the JMAP estimation, while achieving a cost function with a unique global solution, by assuming a gamma prior distribution for the hyperparameters. The resulting fidelity is similar to the quality provided by classical methods such as GCV, L-curve and Evidence, which are computationally expensive. Experimental results illustrate the low complexity and stability of the proposed method.