Bayesian Image Research Articles

Radiation image reconstruction and uncertainty quantification using a Gaussian process prior

We propose a complete framework for Bayesian image reconstruction and uncertainty quantification based on a Gaussian process prior (GPP) to overcome limitations of maximum likelihood expectation maximization (ML-EM) image reconstruction algorithm. The prior distribution is constructed with a zero-mean Gaussian process (GP) with a choice of a covariance function, and a link function is used to map the Gaussian process to an image. Unlike many other maximum a posteriori approaches, our method offers highly interpretable hyperparamters that are selected automatically with the empirical Bayes method. Furthermore, the GP covariance function can be modified to incorporate a priori structural priors, enabling multi-modality imaging or contextual data fusion. Lastly, we illustrate that our approach lends itself to Bayesian uncertainty quantification techniques, such as the preconditioned Crank–Nicolson method and the Laplace approximation. The proposed framework is general and can be employed in most radiation image reconstruction problems, and we demonstrate it with simulated free-moving single detector radiation source imaging scenarios. We compare the reconstruction results from GPP and ML-EM, and show that the proposed method can significantly improve the image quality over ML-EM, all the while providing greater understanding of the source distribution via the uncertainty quantification capability. Furthermore, significant improvement of the image quality by incorporating a structural prior is illustrated.

Bayesian self-calibration and imaging in very long baseline interferometry

Context. Self-calibration methods with the CLEAN algorithm have been widely employed in very long baseline interferometry (VLBI) data processing in order to correct antenna-based amplitude and phase corruptions present in the data. However, human interaction during the conventional CLEAN self-calibration process can impose a strong effective prior, which in turn may produce artifacts within the final image and hinder the reproducibility of final results. Aims. In this work, we aim to demonstrate a combined self-calibration and imaging method for VLBI data in a Bayesian inference framework. The method corrects for amplitude and phase gains for each antenna and polarization mode by inferring the temporal correlation of the gain solutions. Methods. We use Stokes I data of M87 taken with the Very Long Baseline Array (VLBA) at43 GHz, pre-calibrated using the rPICARD CASA-based pipeline. For antenna-based gain calibration and imaging, we use the Bayesian imaging software resolve. To estimate gain and image uncertainties, we use a variational inference method. Results. We obtain a high-resolution M87 Stokes I image at 43 GHz in conjunction with antenna-based gain solutions using our Bayesian self-calibration and imaging method. The core with counter-jet structure is better resolved, and extended jet emission is better described compared to the CLEAN reconstruction. Furthermore, uncertainty estimation of the image and antenna-based gains allows us to quantify the reliability of the result. Conclusions. Our Bayesian self-calibration and imaging method is able to reconstruct robust and reproducible Stokes I images and gain solutions with uncertainty estimation by taking into account the uncertainty information in the data.

Bayesian image segmentation under varying blur with triplet Markov random field

Abstract In this paper, we place ourselves in the context of the Bayesian framework for image segmentation in the presence of varying blur. The proposed approach is based on Triplet Markov Random Fields (TMRF). This method takes into account, during segmentation, peculiarities of an image such as noise, blur, and texture. We present an unsupervised TMRF method, which jointly deals with the problem of segmentation, and that of depth estimation in order to process fluorescence microscopy images. In addition to the estimation of the depth maps using the Metropolis-Hasting and the Stochastic Parameter Estimation (SPE) algorithms, we also estimate the model parameters using the SPE algorithm. We compare our TMRF method to other MRF models on simulated images, and to an unsupervised method from the state of art on real fluorescence microscopy images. Our method offers improved results, especially when blur is important.

Bayesian Constraints on the Ring Ellipticity of M87* 2017 Using Themis

Measuring the properties of black hole images has the potential to constrain deviations from general relativity on horizon scales. Of particular interest is the ellipticity of the ring that is sensitive to the underlying spacetime. In 2019, the Event Horizon Telescope (EHT) produced the first-ever image of a black hole on horizon scales. Here, we reanalyze the M87* EHT 2017 data using Bayesian imaging (BI) techniques, constructing a posterior of the ring shape. We find that BI recovers the true on-sky ring shape more reliably than the original imaging methods used in 2019. As a result, we find that M87*'s ring ellipticity is 0.09−0.06+0.07 and is consistent with the measured ellipticity from general relativistic magnetohydrodynamic simulations.

Bayes R-CNN: An Uncertainty-Aware Bayesian Approach to Object Detection in Remote Sensing Imagery for Enhanced Scene Interpretation

Remote sensing technology has been modernized by artificial intelligence, which has made it possible for deep learning algorithms to extract useful information from images. However, overfitting and lack of uncertainty quantification, high-resolution images, information loss in traditional feature extraction, and background information retrieval for detected objects limit the use of deep learning models in various remote sensing applications. This paper proposes a Bayes by backpropagation (BBB)-based system for scene-driven identification and information retrieval in order to overcome the above-mentioned problems. We present the Bayes R-CNN, a two-stage object detection technique to reduce overfitting while also quantifying uncertainty for each object recognized within a given image. To extract features more successfully, we replace the traditional feature extraction model with our novel Multi-Resolution Extraction Network (MRENet) model. We propose the multi-level feature fusion module (MLFFM) in the inner lateral connection and a Bayesian Distributed Lightweight Attention Module (BDLAM) to reduce information loss in the feature pyramid network (FPN). In addition, our system incorporates a Bayesian image super-resolution model which enhances the quality of the image to improve the prediction accuracy of the Bayes R-CNN. Notably, MRENet is used to classify the background of the detected objects to provide detailed interpretation of the object. Our proposed system is comprehensively trained and assessed utilizing the state-of-the-art DIOR and HRSC2016 datasets. The results demonstrate our system’s ability to detect and retrieve information from remote sensing scene images.

Implementation a Different Segmentation Method as a Joint Prior Model to Reconstruction the Bayesian Image analysis

The basic concepts in the application of Bayesian approaches to image analysis have been introduced as advanced method. The Bayesian approach contains benefits in respect to image analysis and interpretation because it permits the use of prior knowledge concerning the situation under study. This paper use to investigate the application of some of the well-known procedures (determines number labels of image under several conditions like noise of image, resolution of image) for the Bayesian image analysis with segmentation as a joint prior model in order to estimate the Maximum Likelihood (ML). Markov random field with segmentation which resulted by mean of posterior (MP). This paper contains several sections. Firstly, includes introduction about the image analysis with, Bayes frame work and statistical background to Markov's random field and its relationship through Markov Chain Monte Carlo. Secondly, section, which directly addresses solutions by using a principle of segmentation, which is a representation in threshold, is simply type introduced. Thirdly, presents the description of Experiment of Segmentation by depend on histogram also study of the factor (one prior) and the same model by adding segmentation (joint prior) based on techniques presented in the previously section are discussed . Fourthly, this section contains on the second prior implementation and simulation by using phantom data (Castle from south East Asia) also steps of estimation. Finally, result of experiment as well as estimation and summary.

Accelerated Bayesian Imaging by Relaxed Proximal-Point Langevin Sampling

Accelerated Bayesian Imaging by Relaxed Proximal-Point Langevin Sampling

A data-driven approach for the partial reconstruction of individual human molar teeth using generative deep learning.

Due to the high prevalence of dental caries, fixed dental restorations are regularly required to restore compromised teeth or replace missing teeth while retaining function and aesthetic appearance. The fabrication of dental restorations, however, remains challenging due to the complexity of the human masticatory system as well as the unique morphology of each individual dentition. Adaptation and reworking are frequently required during the insertion of fixed dental prostheses (FDPs), which increase cost and treatment time. This article proposes a data-driven approach for the partial reconstruction of occlusal surfaces based on a data set that comprises 92 3D mesh files of full dental crown restorations. A Generative Adversarial Network (GAN) is considered for the given task in view of its ability to represent extensive data sets in an unsupervised manner with a wide variety of applications. Having demonstrated good capabilities in terms of image quality and training stability, StyleGAN-2 has been chosen as the main network for generating the occlusal surfaces. A 2D projection method is proposed in order to generate 2D representations of the provided 3D tooth data set for integration with the StyleGAN architecture. The reconstruction capabilities of the trained network are demonstrated by means of 4 common inlay types using a Bayesian Image Reconstruction method. This involves pre-processing the data in order to extract the necessary information of the tooth preparations required for the used method as well as the modification of the initial reconstruction loss. The reconstruction process yields satisfactory visual and quantitative results for all preparations with a root mean square error (RMSE) ranging from 0.02 mm to 0.18 mm. When compared against a clinical procedure for CAD inlay fabrication, the group of dentists preferred the GAN-based restorations for 3 of the total 4 inlay geometries. This article shows the effectiveness of the StyleGAN architecture with a downstream optimization process for the reconstruction of 4 different inlay geometries. The independence of the reconstruction process and the initial training of the GAN enables the application of the method for arbitrary inlay geometries without time-consuming retraining of the GAN.

Bayesian imaging inverse problem with SA-Roundtrip prior via HMC-pCN sampler

Bayesian inference with deep generative prior has received considerable interest for solving imaging inverse problems in many scientific and engineering fields. The selection of the prior distribution is learned from, and therefore an important representation learning of, available prior measurements. The SA-Roundtrip, a novel deep generative prior, is introduced to enable controlled sampling generation and identify the data's intrinsic dimension. This prior incorporates a self-attention structure within a bidirectional generative adversarial network. Subsequently, Bayesian inference is applied to the posterior distribution in the low-dimensional latent space using the Hamiltonian Monte Carlo with preconditioned Crank-Nicolson (HMC-pCN) algorithm, which is proven to be ergodic under specific conditions. Experiments conducted on computed tomography (CT) reconstruction with the MNIST and TomoPhantom datasets reveal that the proposed method outperforms state-of-the-art comparisons, consistently yielding a robust and superior point estimator along with precise uncertainty quantification.

The first spatio-spectral Bayesian imaging of SN1006 in X-rays

Supernovae (SNs) are an important source of energy in the interstellar medium. Young remnants of supernovae (SNRs) exhibit peak emission in the X-ray region, making them interesting objects for X-ray observations. In particular, the supernova remnant SN1006 is of great interest due to its historical record, proximity, and brightness. Thus, it has been studied with a number of X-ray telescopes. Improving X-ray imaging of this and other remnants is an important but challenging task, as it often requires multiple observations with different instrument responses to image the entire object. Here, we use Chandra observations to demonstrate the capabilities of Bayesian image reconstruction using information field theory (IFT). Our objective is to reconstruct denoised, deconvolved, and spatio-spectral resolved images from X-ray observations and to decompose the emission into different morphologies, namely, diffuse and point-like. Further, we aim to fuse data from different detectors and pointings into a mosaic and quantify the uncertainty of our result. By utilizing prior knowledge on the spatial and spectral correlation structure of the diffuse emission and point sources, this method allows for the effective decomposition of the signal into these two components. In order to accelerate the imaging process, we introduced a multi-step approach, in which the spatial reconstruction obtained for a single energy range is used to derive an informed starting point for the full spatio-spectral reconstruction. We applied this method to 11 Chandra observations of SN1006 from 2008 and 2012, providing a detailed, denoised, and decomposed view of the remnant. In particular, the separated view of the diffuse emission ought to provide new insights into the complex, small-scale structures in the center of the remnant and at the shock front profiles. For example, our analysis reveals sharp X-ray flux increases by up to two orders of magnitude at the shock fronts of SN1006.

Proximal MCMC for Bayesian Inference of Constrained and Regularized Estimation

This article advocates proximal Markov chain Monte Carlo (ProxMCMC) as a flexible and general Bayesian inference framework for constrained or regularized estimation. Originally introduced in the Bayesian imaging literature, ProxMCMC employs the Moreau-Yosida envelope for a smooth approximation of the total-variation regularization term, fixes variance and regularization strength parameters as constants, and uses the Langevin algorithm for the posterior sampling. We extend ProxMCMC to be fully Bayesian by providing data-adaptive estimation of all parameters including the regularization strength parameter. More powerful sampling algorithms such as Hamiltonian Monte Carlo are employed to scale ProxMCMC to high-dimensional problems. Analogous to the proximal algorithms in optimization, ProxMCMC offers a versatile and modularized procedure for conducting statistical inference on constrained and regularized problems. The power of ProxMCMC is illustrated on various statistical estimation and machine learning tasks, the inference of which is traditionally considered difficult from both frequentist and Bayesian perspectives.

Empirical Bayesian Imaging With Large-Scale Push-Forward Generative Priors

Empirical Bayesian Imaging With Large-Scale Push-Forward Generative Priors

Contributed Session II: Computational modeling of shift in unique yellow for small stimuli.

Unique yellow (UY) is largely invariant to L:M cone proportion for spatially-extended stimuli in healthy trichromats. However, a recent adaptive-optics-based study by Boehm et al. reveals that when stimulus size is reduced to a few arcmin, color appearance depends on the local L:M proportion in the patch of the retina on which the stimulus was imaged. We aimed to determine if such findings are consistent with a normative account of visual processing. A series of 3.5 and 10 arcmin stimuli were simulated as isoluminant mixtures of 540 and 680 nm primaries. We modeled sensory encoding under adaptive-optics conditions using the open-source software ISETBio, for simulated retinal cone mosaics with varying local L:M proportions. The resultant cone excitations were decoded using a Bayesian image reconstruction algorithm (Zhang et al., 2022). For the 3.5 arcmin stimuli, as local L:M proportion decreased, the 540 nm component of the reconstructions increased relative to the 680 nm component. This is qualitatively consistent with the experimental observations of Boehm et al. For 10 arcmin stimuli, in contrast, reconstructions were stable across variation in local L:M cone proportion. Notably, reconstructions depend not only on the local L:M cone proportion, but also on the proportion in the immediately surrounding retina, leading to a testable prediction. The computational observations frame the experimental results as a normative consequence of visual processing.

Bayesian radio interferometric imaging with direction-dependent calibration

Context. Radio interferometers measure frequency components of the sky brightness, modulated by the gains of the individual radio antennas. Due to atmospheric turbulence and variations in the operational conditions of the antennas, these gains fluctuate. Thereby the gains do not only depend on time, but also on the spatial direction on the sky. To recover high-quality radio maps, an accurate reconstruction of the direction and time-dependent individual antenna gains is required. Aims. This paper aims to improve the reconstruction of radio images, by introducing a novel joint imaging and calibration algorithm including direction-dependent antenna gains. Methods. Building on the resolve framework, we designed a Bayesian imaging and calibration algorithm utilizing the image domain gridding method for numerically efficient application of direction-dependent antenna gains. Furthermore, by approximating the posterior probability distribution with variational inference, our algorithm can provide reliable uncertainty maps. Results. We demonstrate the ability of the algorithm to recover high resolution high dynamic range radio maps from VLA data of the radio galaxy Cygnus A. We compare the quality of the recovered images with previous work relying on classically calibrated data. Furthermore, we compare the results with a compressed sensing algorithm also incorporating direction-dependent gains. Conclusions. Including direction-dependent effects in the calibration model significantly improves the dynamic range of the reconstructed images compared to reconstructions from classically calibrated data. Compared to the compressed sensing reconstruction, the resulting sky images have a higher resolution and show fewer artifacts. For utilizing the full potential of radio interferometric data, it is essential to consider the direction dependence of the antenna gains.

Rapid single-photon color imaging of moving objects.

This paper outlines an experimental demonstration of a Bayesian image reconstruction approach to achieve rapid single-photon color imaging of moving objects. The capacity to extract the color of objects is important in a variety of target identification and computer vision applications. Nonetheless, it remains challenging to achieve high-speed color imaging of moving objects in low-photon flux environments. The low-photon regime presents particular challenges for efficient spectral separation and identification, while unsupervised image reconstruction algorithms are often slow and computationally expensive. In this paper, we address both of these difficulties using a combination of hardware and computational solutions. We demonstrate color imaging using a Single-Photon Avalanche Diode (SPAD) detector array for rapid, low-light-level data acquisition, with an integrated color filter array (CFA) for efficient spectral unmixing. High-speed image reconstruction is achieved using a bespoke Bayesian algorithm to produce high-fidelity color videos. The analysis is conducted first on simulated data allowing different pixel formats and photon flux scenarios to be investigated. Experiments are then performed using a plasmonic metasurface-based CFA, integrated with a 64 × 64 pixel format SPAD array. Passive imaging is conducted using white-light illumination of multi-colored, moving targets. Intensity information is recorded in a series of 2D photon-counting SPAD frames, from which accurate color information is extracted using the fast Bayesian method introduced herein. The per-frame reconstruction rate proves to be hundreds of times faster than the previous computational method. Furthermore, this approach yields additional information in the form of uncertainty measures, which can be used to assist with imaging system optimization and decision-making in real-world applications. The techniques demonstrated point the way towards rapid video-rate single-photon color imaging. The developed Bayesian algorithm, along with more advanced SPAD technology and utilization of time-correlated single-photon counting (TCSPC) will permit live 3D, color videography in extremely low-photon flux environments.

Examination of nucleon distribution with Bayesian imaging for isobar collisions

Examination of nucleon distribution with Bayesian imaging for isobar collisions

A novel Bayesian image despeckling method using 2D CGARCH-M model in 2D dost framework

Image denoising is a crucial task in the field of image processing, and Bayesian estimation has emerged as a prominent approach. To effectively employ Bayesian estimation, it is essential to assume a priori distribution corresponding to the transform coefficients of the noise-free image. In this paper, we propose a novel Bayesian image despeckling method that leverages the 2D Complex Generalized Autoregressive Conditional Heteroscedasticity Mixture (CGARCH-M) model within the framework of the 2D Discrete Orthonormal Stockwell Transform (DOST). Prior to introducing this method, we present a novel statistical analysis of the 2D DOST coefficients of log-transformed images. Speckle noise, commonly found in synthetic aperture radar (SAR) images, is modeled as multiplicative noise. To effectively suppress speckle noise, our proposed method utilizes a novel adaptive Bayes risk estimator known as compound maximum a posteriori (CMAP). By employing CMAP, we estimate the noise-free 2D DOST coefficients from the noisy ones, which are modeled using the 2D CGARCH-M. Notably, this paper introduces the 2D CGARCH-M model for 2D complex stochastic processes for the first time, extending the capabilities of the GARCH model and its subsequent extensions that were limited to real-valued processes. The proposed model incorporates location-dependent conditional variances to capture the non-Gaussian statistics of 2D DOST coefficients of log-transformed images and the dependencies between them. Through our statistical analysis, we establish the compatibility between the 2D CGARCH-M model and these coefficients. Our proposed method takes into account both magnitude and phase by utilizing the real and imaginary components of the DOST coefficients concurrently. It provides an optimal and closed-form solution, significantly reducing memory and computational requirements. Moreover, unlike state-of-the-art approaches in this domain, our method exhibits robustness to initial parameter settings. To evaluate the effectiveness of our approach, we conduct comparisons with other denoising methods on artificially speckled aerial images and actual SAR images. The results demonstrate the superior performance of our method.

Butterfly Transforms for Efficient Representation of Spatially Variant Point Spread Functions in Bayesian Imaging

Bayesian imaging algorithms are becoming increasingly important in, e.g., astronomy, medicine and biology. Given that many of these algorithms compute iterative solutions to high-dimensional inverse problems, the efficiency and accuracy of the instrument response representation are of high importance for the imaging process. For efficiency reasons, point spread functions, which make up a large fraction of the response functions of telescopes and microscopes, are usually assumed to be spatially invariant in a given field of view and can thus be represented by a convolution. For many instruments, this assumption does not hold and degrades the accuracy of the instrument representation. Here, we discuss the application of butterfly transforms, which are linear neural network structures whose sizes scale sub-quadratically with the number of data points. Butterfly transforms are efficient by design, since they are inspired by the structure of the Cooley-Tukey fast Fourier transform. In this work, we combine them in several ways into butterfly networks, compare the different architectures with respect to their performance and identify a representation that is suitable for the efficient representation of a synthetic spatially variant point spread function up to a 1% error. Furthermore, we show its application in a short synthetic example.

Bayesian Image Super-Resolution With Deep Modeling of Image Statistics.

Modeling statistics of image priors is useful for image super-resolution, but little attention has been paid from the massive works of deep learning-based methods. In this work, we propose a Bayesian image restoration framework, where natural image statistics are modeled with the combination of smoothness and sparsity priors. Concretely, first we consider an ideal image as the sum of a smoothness component and a sparsity residual, and model real image degradation including blurring, downscaling, and noise corruption. Then, we develop a variational Bayesian approach to infer their posteriors. Finally, we implement the variational approach for single image super-resolution (SISR) using deep neural networks, and propose an unsupervised training strategy. The experiments on three image restoration tasks, i.e., ideal SISR, realistic SISR, and real-world SISR, demonstrate that our method has superior model generalizability against varying noise levels and degradation kernels and is effective in unsupervised SISR. The code and resulting models are released via https://zmiclab.github.io/projects.html.

A Joint-Parameter Estimation and Bayesian Reconstruction Approach to Low-Dose CT.

Most penalized maximum likelihood methods for tomographic image reconstruction based on Bayes' law include a freely adjustable hyperparameter to balance the data fidelity term and the prior/penalty term for a specific noise-resolution tradeoff. The hyperparameter is determined empirically via a trial-and-error fashion in many applications, which then selects the optimal result from multiple iterative reconstructions. These penalized methods are not only time-consuming by their iterative nature, but also require manual adjustment. This study aims to investigate a theory-based strategy for Bayesian image reconstruction without a freely adjustable hyperparameter, to substantially save time and computational resources. The Bayesian image reconstruction problem is formulated by two probability density functions (PDFs), one for the data fidelity term and the other for the prior term. When formulating these PDFs, we introduce two parameters. While these two parameters ensure the PDFs completely describe the data and prior terms, they cannot be determined by the acquired data; thus, they are called complete but unobservable parameters. Estimating these two parameters becomes possible under the conditional expectation and maximization for the image reconstruction, given the acquired data and the PDFs. This leads to an iterative algorithm, which jointly estimates the two parameters and computes the to-be reconstructed image by maximizing a posteriori probability, denoted as joint-parameter-Bayes. In addition to the theoretical formulation, comprehensive simulation experiments are performed to analyze the stopping criterion of the iterative joint-parameter-Bayes method. Finally, given the data, an optimal reconstruction is obtained without any freely adjustable hyperparameter by satisfying the PDF condition for both the data likelihood and the prior probability, and by satisfying the stopping criterion. Moreover, the stability of joint-parameter-Bayes is investigated through factors such as initialization, the PDF specification, and renormalization in an iterative manner. Both phantom simulation and clinical patient data results show that joint-parameter-Bayes can provide comparable reconstructed image quality compared to the conventional methods, but with much less reconstruction time. To see the response of the algorithm to different types of noise, three common noise models are introduced to the simulation data, including white Gaussian noise to post-log sinogram data, Poisson-like signal-dependent noise to post-log sinogram data and Poisson noise to the pre-log transmission data. The experimental outcomes of the white Gaussian noise reveal that the two parameters estimated by the joint-parameter-Bayes method agree well with simulations. It is observed that the parameter introduced to satisfy the prior's PDF is more sensitive to stopping the iteration process for all three noise models. A stability investigation showed that the initial image by filtered back projection is very robust. Clinical patient data demonstrated the effectiveness of the proposed joint-parameter-Bayes and stopping criterion.