Maximum A Posterior Research Articles

On the effectiveness of partially deterministic Bayesian neural networks

AbstractBayesian neural networks (BNNs) with computationally expensive Hamiltonian Monte Carlo sampling methods are often considered to provide better predictive performance than the maximum a posterior (MAP) solution. Here, as an alternative to sampling all parameters of a BNN (full-random), we experimentally evaluate partially deterministic BNNs that fix some part of the neural network parameters to their MAP solution. In particular, we consider various strategies for fixing half, or all parameters of a layer to the MAP-solution. Over a wide variety of regression and classification tasks, we find that partially deterministic BNNs often significantly improve predictive performance over the MAP-solution, with up to around 24% reduction in negative log-likelihood. Notably, we also find that partially deterministic BNNs that fix half of the parameters in each layer can also reduce under-fitting of full-random BNNs, resulting in up to 7% reduction in negative log-likelihood.

Imaging quality enhancement in photon-counting single-pixel imaging via an ADMM-based deep unfolding network in small animal fluorescence imaging.

Photon-counting single-pixel imaging (SPI) can image under low-light conditions with high-sensitivity detection. However, the imaging quality of these systems will degrade due to the undersampling and intrinsic photon-noise in practical applications. Here, we propose a deep unfolding network based on the Bayesian maximum a posterior (MAP) estimation and alternating direction method of multipliers (ADMM) algorithm. The reconstruction framework adopts a learnable denoiser by convolutional neural network (CNN) instead of explicit function with hand-crafted prior. Our method enhances the imaging quality compared to traditional methods and data-driven CNN under different photon-noise levels at a low sampling rate of 8%. Using our method, the sensitivity of photon-counting SPI prototype system for fluorescence imaging can reach 7.4 pmol/ml. In-vivo imaging of a mouse bearing tumor demonstrates an 8-times imaging efficiency improvement.

Bottom Multi-Parameter Bayesian Inversion Based on an Acoustic Backscattering Model

The geoacoustic and physical properties of the bottom, as well as spatial distribution, are crucial factors in analyzing the underwater acoustic field structure and establishing a geoacoustic model. Acoustic inversion has been widely used as an economical and effective method to obtain multi-parameters of the bottom. Compared with traditional inversion methods based on acoustic propagation models, acoustic backscattering models are more suitable for multi-parameter inversion, because they contain more bottom information. In this study, a Bayesian inversion method based on an acoustic backscattering model is proposed to obtain bottom multi-parameters, including geoacoustic parameters (the sound speed and loss parameter), partial physical parameters of the sediment, and statistical parameters of the seafloor roughness and sediment heterogeneity. The bottom was viewed as a kind of fluid medium. A high-frequency backscattering model based on fluid theory was adopted as the forward model to fit the scattering strength between the model prediction and the measured data. The Bayesian inversion method was used to obtain the posterior probability density (PPD) of the inversion parameters. Parameter estimation, uncertainty, and correlation were acquired by calculating the maximum a posterior (MAP), the mean values, the one-dimensional marginal distributions of the PPD, and the covariance matrix. Finally, the high-frequency bottom backscattering strength from the Quinault Range site was employed for inversion tests. The estimated values and uncertainties of various bottom parameters are presented and compared with the directly measured bottom parameters. The comparison results demonstrate that the method proposed herein can be used to estimate the sediment/water sound speed ratio, the sediment/water density ratio, and the spectral exponent of the roughness spectrum effectively and reliably.

Underlying Topography Estimation over Forest Using Maximum a Posteriori Inversion with Spaceborne Polarimetric SAR Interferometry

This paper presents a method for extracting the digital elevation model (DEM) of forested areas from polarimetric interferometric synthetic aperture radar (PolInSAR) data. The method models the ground phase as a Von Mises distribution, with a mean of the topographic phase computed from an external DEM. By combining the prior distribution of the ground phase with the complex Wishart distribution of the observation covariance matrix, we derive the maximum a posterior (MAP) inversion method based on the RVoG model and analyze its Cramer–Rao Lower Bound (CRLB). Furthermore, considering the characteristics of the objective function, this paper introduces a Four-Step Optimization (FSO) method based on gradient optimization, which solves the inefficiency problem caused by exhaustive search in solving ground phase using the MAP method. The method is validated using spaceborne L-band repeat-pass SAOCOM data from a test forest area. The test results for FSO indicate that it is approximately 5.6 times faster than traditional methods without compromising accuracy. Simultaneously, the experimental results demonstrate that the method effectively solves the problem of elevation jumps in DEM inversion when modeling the ground phase with the Gaussian distribution. ICESAT-2 data are used to evaluate the accuracy of the inverted DEM, revealing that our method improves the root mean square error (RMSE) by about 23.6% compared to the traditional methods.

On Finite and Non-Finite Bayesian Mixture Models

In this paper, a Bayesian paradigm of a mixture model with finite and non-finite components is expounded for a generic prior and likelihood that can be of any distributional random noise. The mixture model consists of stylized properties-proportional allocation, sample size allocation, and latent (unobserved) variable for similar probabilistic generalization. The Expectation-Maximization (EM) algorithm technique of parameter estimation was adopted to estimate the stated stylized parameters. The Markov Chain Monte Carlo (MCMC) and Metropolis–Hastings sampler algorithms were adopted as an alternative to the EM algorithm when it is not analytically feasible, that is, when the unobserved variable cannot be replaced by imposed expectations (means) and when there is need for correction of exploration of posterior distribution by means of acceptance ratio quantity, respectively. Label switching for exchangeability of posterior distribution via truncated or alternating prior distributional form was imposed on the posterior distribution for robust tailoring inference through Maximum a Posterior (MAP) index. In conclusion, it was deduced via simulation study that the number of components grows large for all permutations to be considered for subsample permutations.

Reasoning Disaster Chains with Bayesian Network Estimated Under Expert Prior Knowledge

With the acceleration of global climate change and urbanization, disaster chains are always connected to artificial systems like critical infrastructure. The complexity and uncertainty of the disaster chain development process and the severity of the consequences have brought great challenges to emergency decision makers. The Bayesian network (BN) was applied in this study to reason about disaster chain scenarios to support the choice of appropriate response strategies. To capture the interacting relationships among different factors, a scenario representation model of disaster chains was developed, followed by the determination of the BN structure. In deriving the conditional probability tables of the BN model, we found that, due to the lack of data and the significant uncertainty of disaster chains, parameter learning methodologies based on data or expert knowledge alone are insufficient. By integrating both sample data and expert knowledge with the maximum entropy principle, we proposed a parameter estimation algorithm under expert prior knowledge (PEUK). Taking the rainstorm disaster chain as an example, we demonstrated the superiority of the PEUK-built BN model over the traditional maximum a posterior (MAP) algorithm and the direct expert opinion elicitation method. The results also demonstrate the potential of our BN scenario reasoning paradigm to assist real-world disaster decisions.

Understanding Deep MIMO Detection

Incorporating deep learning (DL) into multiple-input multiple-output (MIMO) detection has been deemed as a promising technique for future wireless communications. However, most of the DL-based MIMO detection algorithms are lack of interpretation on internal mechanisms. In this paper, we analyze the performance of the DL-based MIMO detection to better understand its strengths and weaknesses. We investigate and compare two different models: data-driven DL detector with neural networks activated by rectifier linear unit (ReLU) function and model-driven DL detector based on traditional detection algorithms. We show that the data-driven DL detector asymptotically approaches to the maximum a posterior (MAP) detector in various scenarios but requires a large amount of training samples to converge in time-varying channels. On the other hand, the model-driven DL detector utilizes the expert knowledge to alleviate the impact of channels and achieves relatively high detection accuracy with a small set of training data. Simulation results confirm our analytical results and demonstrate the effectiveness of the DL-based MIMO detection for both linear and nonlinear signal systems.

Localization and Motion Planning of Industrial Tractor–Trailers Vehicles

This brief presents real-time configuration estimation and motion planning for the industrial tractor–trailers vehicle composed of a full-scale car-like tractor and multiple full trailers. For the real-life vehicle, determining the configuration is challenging. With only on-tractor sensors, we solve this problem by fusing information from system dynamics propagation, geometrical constraints among the articulated units, and matching with the prebuilt environment map. The solution is efficiently achieved by formulating and solving a maximum a posterior (MAP) estimation problem in the pose-graph optimization framework. With the complicated tractor–trailers’ structure, small mismatch between actual and planned trajectories is crucial to inherit obstacle-free guarantee from planning to execution. We consider the dynamics and focus on facilitating the reduction of the mismatch by proposing a controller-based smoothing method to perform online motion planning. The given waypoint path is smoothed by forward propagation using a deliberately designed controller to generate the trajectory. The efficiently computed trajectories are also obstacle-free and dynamically feasible. The controller is also applied in execution to precisely reproduce the planned system evolvement. We demonstrate the performance with the real-life industrial tractor–trailers’ vehicle.

Information Functions of Rank‐2PL Models for Forced‐Choice Questionnaires

AbstractThis paper presents the item and test information functions of the Rank two‐parameter logistic models (Rank‐2PLM) for items with two (pair) and three (triplet) statements in forced‐choice questionnaires. The Rank‐2PLM model for pairs is the MUPP‐2PLM (Multi‐Unidimensional Pairwise Preference) and, for triplets, is the Triplet‐2PLM. Fisher's information and directional information are described, and the test information for Maximum Likelihood (ML), Maximum A Posterior (MAP), and Expected A Posterior (EAP) trait score estimates is distinguished. Expected item/test information indexes at various levels are proposed and plotted to provide diagnostic information on items and tests. The expected test information indexes for EAP scores may be difficult to compute due to a typical test's vast number of item response patterns. The relationships of item/test information with discrimination parameters of statements, standard error, and reliability estimates of trait score estimates are discussed and demonstrated using real data. Practical suggestions for checking the various expected item/test information indexes and plots are provided.

A dual adaptive robust control for nonlinear systems with parameter and state estimation

Stabilization and learning are imperative to the high-performance feedback control of nonlinear systems. A dual adaptive robust control (DARC) scheme is proposed for nonlinear systems with model uncertainties to achieve a desired level of performance. Only the output of the nonlinear system is accessible in this work, all the states and parameters are learned online. Firstly, the DARC uses the prior physical bounds of systems to design a discontinuous projection with update rate limits which confines the bounds of parameter and state estimation. Then robustness of the nonlinear system can be guaranteed by the deterministic robust control (DRC) method. Secondly, a dual adaptive estimation mechanism (DAEM) is developed to learn the unknown parameters and states of systems. One part of the DAEM is the bounded gain forgetting (BGF) estimator, which is developed to handle inaccurate parameters and parametric variations. The other is the adaptive unscented Kalman filter (AUKF) synthesized for state estimation. The AUKF contains a statistic estimator based on the maximum a posterior (MAP) rule to estimate the unknown covariance matrix. Finally, simulation results illustrate the effectiveness of the suggested method.

Deep Gaussian Scale Mixture Prior for Image Reconstruction.

Image reconstruction from partial observations has attracted increasing attention. Conventional image reconstruction methods with hand-crafted priors often fail to recover fine image details due to the poor representation capability of the hand-crafted priors. Deep learning methods attack this problem by directly learning mapping functions between the observations and the targeted images can achieve much better results. However, most powerful deep networks lack transparency and are nontrivial to design heuristically. This paper proposes a novel image reconstruction method based on the Maximum a Posterior (MAP) estimation framework using learned Gaussian Scale Mixture (GSM) prior. Unlike existing unfolding methods that only estimate the image means (i.e., the denoising prior) but neglected the variances, we propose characterizing images by the GSM models with learned means and variances through a deep network. Furthermore, to learn the long-range dependencies of images, we develop an enhanced variant based on the Swin Transformer for learning GSM models. All parameters of the MAP estimator and the deep network are jointly optimized through end-to-end training. Extensive simulation and real data experimental results on spectral compressive imaging and image super-resolution demonstrate that the proposed method outperforms existing state-of-the-art methods.

Deep Unfolding Network for Efficient Mixed Video Noise Removal

Existing image and video denoising algorithms have focused on removing homogeneous Gaussian noise. However, this assumption with noise modeling is often too simplistic for the characteristics of real-world noise. Moreover, the design of network architectures in most deep learning-based video denoising methods is heuristic, ignoring valuable domain knowledge. In this paper, we propose a model-guided deep unfolding network for the more challenging and realistic mixed noise video denoising problem, named DU-MVDnet. First, we develop a novel observation model/likelihood function based on the correlations among adjacent degraded frames. In the framework of Bayesian deep learning, we introduce a deep image denoiser prior and obtain an iterative optimization algorithm based on the maximum a posterior (MAP) estimation. To facilitate end-to-end optimization, the iterative algorithm is transformed into a deep convolutional neural network (DCNN)-based implementation. Furthermore, recognizing the limitations of traditional motion estimation and compensation methods, we propose an efficient multistage recursive fusion strategy to exploit temporal dependencies. Specifically, we divide video frames into several overlapping groups and progressively integrate these frames into one frame. Toward this objective, we implement a multiframe adaptive aggregation operation to integrate feature maps of intragroup with those of intergroup frames. Extensive experimental results on different video test datasets have demonstrated that the proposed model-guided deep network outperforms current state-of-the-art video denoising algorithms such as FastDVDnet and MAP-VDNet.

Image restoration for optical synthetic aperture system via variational physics-informed network

Optical synthetic aperture with homogeneous circular sub-mirrors greatly improves the spatial resolution of space telescopes; however, the discrete and sparse characteristics of the sub-mirrors reduce the mid-frequency modulation transfer function (MTF), resulting in blurred images being obtained. In this paper, a method combining variational physics-informed with deep learning is presented, which shows blind image restoration without complex priors. The constraint effect of traditional maximum a posterior (MAP) framework is removed by variational inference framework, which is embedded into Variational Physics-informed Network (VPIN) to optimize neural network training. Residual dense blocks (RDBs) construction is contributed to image feature extraction. Networks with SSIM-corrected loss functions can be trained at the feature level to help with convergence. When SNR = 30 dB, the PSNR of Golay-6 remote sensing test set increases from 20.16 dB to 23.90 dB, SSIM is from 0.610 to 0.842, and MS-SSIM is from 0.930 to 0.955.

Fundus image enhancement using visual transformation and maximum a posterior estimation

Fundus imaging is used to diagnose many common diseases in human vision. Because of uneven brightness, blur effect, and low contrast, fundus images are not helpful for diagnosis. The proposed technique provides an effective enhancement technique for fundus images. The method performs the appropriate illumination improvement, details correction, and denoising operations. A maximum a posterior (MAP) estimator is used to calculate the illumination and reflection components, which is motivated by the retinex method. The estimated illumination part is put through gamma correction to smooth its unevenness. A straightforward visual transformation algorithm is applied to address the unequal luminosity at the structural layer; it is used to enhance the details of the structure layer. The suggested methods were examined on public datasets (STARE, CHASEDB1, and DRIVE). To further confirm the recommended strategy's benefit in aiding the diagnosis, ophthalmologists' quality assessments were also taken. The objective evaluation used performance parameters such as GMSD, NIQE, HPSI, VSI, MCSD, and PSNR. According to experimental results, the suggested method may simultaneously manage the duties of illumination enhancement, details improvement, noise, and artifact suppression, outperforming state-of-the-art methods. According to the qualitative, quantitative, and statistical examination, the suggested algorithm reliably improves low-contrast retinal photos with intact color and a natural appearance.

Blind Image Denoising via Deep Unfolding Network With Degradation Information Guidance

Blind denoising is an active research area in image processing. In recent years, denoising methods based on deep learning have achieved outstanding results. However, directly designing more complex networks is very challenging and often lacks interpretability. Besides, existing methods also often ignore the guidance of degradation information. Therefore, how to guide the design of deep neural networks by combining traditional algorithms while predicting and utilizing degradation information is an open problem. In this brief, based on the maximum a posterior (MAP) framework, we first estimate the degradation information, and design corresponding operators to obtain initial restoration estimation in high-dimensional mapping space. On this basis, the initial estimation is substituted into the denoising problem. According to the splitting algorithm and momentum based gradient descent method, the iterative optimization of the substituted problem is carried out, and the proposed algorithm framework is obtained. Furthermore, according to the proposed algorithm framework, we design the corresponding network structure, and a novel blind unfolding network named BDUNet is introduced. Experimental results show that our network not only outperforms blind methods but also has advantages over non-blind methods.

Location Tracking for Reconfigurable Intelligent Surfaces Aided Vehicle Platoons: Diverse Sparsities Inspired Approaches

In this paper, we investigate the employment of reconfigurable intelligent surfaces (RISs) into vehicle platoons, functioning in tandem with a base station (BS) in support of the high-precision location tracking. In particular, the use of a RIS imposes additional structured sparsity that, when paired with the initial sparse line-of-sight (LoS) channels of the BS, facilitates beneficial group sparsity. The resultant group sparsity significantly enriches the energies of the original direct-only channel, enabling a greater concentration of the LoS channel energies emanated from the same vehicle location index. Furthermore, the burst sparsity is exposed by representing the non-line-of-sight (NLoS) channels as their sparse copies. This thus constitutes the philosophy of the diverse sparsities of interest. Then, a diverse dynamic layered structured sparsity (DiLuS) framework is customized for capturing different priors for this pair of sparsities, based upon which the location tracking problem is formulated as a maximum a posterior (MAP) estimate of the location. Nevertheless, the tracking issue is highly intractable due to the ill-conditioned sensing matrix, intricately coupled latent variables associated with the BS and RIS, and the spatial-temporal correlations among the vehicle platoon. To circumvent these hurdles, we propose an efficient algorithm, namely DiLuS enabled spatial-temporal platoon localization (DiLuS-STPL), which incorporates both variational Bayesian inference (VBI) and message passing techniques for recursively achieving parameter updates in a turbo-like way. Finally, we demonstrate through extensive simulation results that the localization relying exclusively upon a BS and a RIS may achieve the comparable precision performance obtained by the two individual BSs, along with the robustness and superiority of our proposed algorithm as compared to various benchmark schemes.

A Spatial–Temporal Bayesian Deep Image Prior Model for Moderate Resolution Imaging Spectroradiometer Temporal Mixture Analysis

Time-series remote sensing images are important in agricultural monitoring and investigation. However, most time-series data with high temporal resolution have the problem of insufficient spatial resolution which cannot meet the requirement of precision agriculture. The unmixing technique can obtain the object abundances with richer spatial information from the coarse-resolution images. Although the unmixing technique is widely used in hyperspectral data, it is insufficiently researched for time-series data. Temporal unmixing extends spectral unmixing to the time domain from the spectral domain, and describes the temporal characteristics rather than the spectral characteristics of different ground objects. Deep learning (DL) techniques have achieved promising performance for the unmixing problem in recent years, but there are still few studies on temporal mixture analysis (TMA), especially in the application of crop phenological monitoring. This paper presents a novel spatial–temporal deep image prior method based on a Bayesian framework (ST-Bdip), which innovatively combines the knowledge-driven TMA model and the DL-driven model. The normalized difference vegetation index (NDVI) time series of moderate resolution imaging spectroradiometer (MODIS) data is used as the object for TMA, while the extracted seasonal crop signatures and the fractional coverages are perceived as the temporal endmembers (tEMs) and corresponding abundances. The proposed ST-Bdip method mainly includes the following contributions. First, a deep image prior model based on U-Net architecture is designed to efficiently learn the spatial context information, which enhances the representation of abundance modeling compared to the traditional non-negative least squares algorithm. Second, The TMA model is incorporated into the U-Net training process to exploit the knowledge in the forward temporal model effectively. Third, the temporal noise heterogeneity in time-series images is considered in the model optimization process. Specifically, the anisotropic covariance matrix of observations from different time dimensions is modeled as a multivariate Gaussian distribution and incorporated into the calculation of the loss function. Fourth, the "purified means" approach is used to further optimize crop tEMs and the corresponding abundances. Finally, the expectation–maximization (EM) algorithm is designed to solve the maximum a posterior (MAP) problem of the model in the Bayesian framework. Experimental results on three synthetic datasets with different noise levels and two real MODIS datasets demonstrate the superiority of the proposed approach in comparison with seven traditional and advanced unmixing algorithms.

Bayesian Direction of Arrival Estimation with Prior Knowledge from Target Tracker

The performance of traditional direction of arrival (DOA) estimation methods always deteriorates at a low signal-to-noise ratio (SNR) or without sufficient observations. This paper investigates the Bayesian DOA estimation problem aided by the prior knowledge from the target tracker. The Bayesian Cramér–Rao lower bounds (CRLB) and the expected CRLB are first derived to evaluate the theoretical performance of Bayesian DOA estimation. Based on the maximum a posterior (MAP) estimator in the Bayesian framework, two methods are proposed. One is a two-step grid search method for a single target DOA case. The other is a gradient-based iterative solution for multiple targets DOA case, which extends the traditional Newton method by incorporating the prior knowledge. We also propose a minimum mean square error (MMSE) estimator using a Monte Carlo method, which requires trading off accuracy against computational complexity. By comparing with the maximum likelihood (ML) estimators and the MUSIC algorithm, the proposed three Bayesian estimators improve the DOA estimation performance in low SNR or with limited snapshots. Moreover, the performance is not affected by the correlation between sources.

SWM-DE: Statistical wavelet model for joint denoising and enhancement for multimodal medical images

Medical images are usually degraded by numerous noises during acquisition or transmission, which often causes low contrast leading to deterioration of image quality. As such, medical image denoising and enhancement has become a paramount routine task. To overcome this problem, we propose a cutting-edge joint statistical and morphological model for the denoising and enhancement operation. Firstly, we propose a statistical model in formulating the marginal distribution of the wavelet coefficients. This model is integrated into a Bayesian inference framework to develop a maximum a posterior (MAP) estimator of the noise-free coefficient. Based on the statistical model, we eliminate the need for noise level estimation, and allows the model to automatically adapts to the observed image data. Secondly, we propose an adjustable morphological reconstruction model to eliminate known and unknown noises associated with medical images, while preserving the image details. After these operations, the image is decomposed into several wavelet subbands to extract the illumination and detail components. The image is then reconstructed based on the inverse wavelet to generate the enhanced noise-free image.Experimental results show that the proposed framework obtained high EME values of 41.04, 48.81, 47.81, and 45.75 for OCTA, FFA, CT, and X-ray imaging modalities, and performs better than the state-of-the-art methods. The proposed algorithm can effectively and efficiently enhance medical images, which will assist the clinicians in disease diagnosis, monitoring, and treatment.

Evolutionary generative adversarial network based end-to-end learning for MIMO molecular communication with drift system

Molecular communication (MC) is a novel paradigm for nano-communication networks. Compared with diffusion-based single-input single-out (SISO) systems, multiple-input multiple-output (MIMO) MC with drift systems can effectively mitigate the negative effects of inter symbol interference (ISI), inter link interference (ILI) and noise, further improving transmission efficiency. The modeling complexity of MIMO MC systems inspires the application of deep learning (DL) techniques to establish end-to-end architectures for signal recovery. However, training of the entire end-to-end system is limited by the unknown channel and small training sample size. In this paper, aiming at signal recovery of the newly developed mathematical MIMO MC with drift system model, a Kullback–Leibler divergence (KLD) evolutionary generative adversarial network (EGAN)-based end-to-end learning method is proposed. The end-to-end architecture can be trained offline with both the sampled and fake signals generated by KLD EGAN, even with a small training sample size, and then used to recover online transmitted signals directly. Besides, two traditional detection algorithms denoted as the maximum a posterior (MAP) detector and fixed threshold (FT) detector, are proposed as well for theoretical comparison. Experiments of the effect of different model parameters on the system performance have been carried out. Results validate the effectiveness and robustness of our proposed method compared to other DL-based methods, including the deep neural networks (DNN)-based, the original GAN-based, and the original EGAN-based, in terms of transmission accuracy.