Image Reconstruction Approach Research Articles

A Deep Learning Approach for Reconstruction in Millimeter-Wave Imaging Systems

In millimeter-wave (MMW) imaging, the objects of interest are oftentimes modeled as 2-D binary (black and white) shapes with white pixels representing the reflecting interior of the object. However, due to the propagation of the scattered waves, the continuous-domain binary images are convolved with a so-called point-spread function (PSF) before being digitized by means of sampling. As the 2-D PSF is both nonseparable and nonvanishing in the case of MMW imaging, exact recovery is quite complicated. In this communication, we propose a deep-learning approach for image reconstruction. We should highlight that the wave scatterings are suitably represented with complex-valued quantities, while standard deep neural networks (DNNs) accept real-valued inputs. To overcome this challenge, we separate the real and imaginary parts as if we had two imaging modalities and concatenate them to form a real-valued input with a larger size. Fortunately, the network automatically learns how to combine the mutual information between these modalities to reconstruct the final image. Among the advantages of the proposed method are improved robustness against additive noise and mismatch errors of imaging frequency and object to antenna distance; indeed, the method works well in wideband imaging scenarios over a wide range of objects to antenna distances even in the presence of high noise levels without requiring a separate calibration stage. We test the method with synthetic data simulated with software as well as real recordings in the laboratory.

Complex Valued Long Short Term Memory Based Architecture for Frequency Domain Photoacoustic Imaging

Frequency domain photoacoustic (FDPA) imaging has great potential in a clinical setting compared to time-domain photoacoustic imaging due to its reduced cost and small form factors. However FDPA system struggles with lower signal to noise ratio, necessitating the need for advanced image reconstruction methods. Most of the image reconstruction approaches in FDPA imaging are based on analytical or model based schemes. Very less emphasis has been placed on developing deep learning based approaches for FDPA imaging. In this work, a image translation network was developed with the ability to directly map from sinogram data (complex-valued) to the initial pressure rise distribution (real-valued) for FD-PA imaging. This architecture was based on a Long Short Term Memory (LSTM) backbone (with adjoined real and imaginary parts of the complex sinogram data as input) followed by a fully connected layer, which is then passed through a convolution and transposed convolution layer pair. The result of the FDPA-LSTM architecture was compared with direct translational networks based on ResNet, UNet and AUTOMAP and found to have an improvement of about 15% in terms of PSNR and 10% in terms of SSIM with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$150^{\circ }$</tex-math></inline-formula> data acquisition limited-view angle. Further, the FDPA-LSTM was also compared with post-processing UNet architecture on backprojection and Tikhonov regularized reconstruction. A 20% improvement in terms of PSNR with backprojection and post-processing UNet was observed. Further FDPA-LSTM had similar performance as Tikhonov and post-processing UNet (with 75 times acceleration). The developed scheme will indeed be very useful for achieving accelerated and accurate frequency domain photoacoustic imaging.

Full Wave Inversion and Inverse Scattering in Ultrasound Tomography/Volography.

Ultrasound breast tomography has been around for more than 40years. Early approaches to reconstruction focused on simple algebraic reconstructions and bent ray techniques. These approaches were not able to provide high-quality and high spatial-resolution images. The advent of inverse scattering approaches resulted in a shift in image reconstruction approaches for breast tomography and a subsequent improvement in image quality. Full wave inverse solvers were developed to improve the reconstruction times without sacrificing image quality. The development of GPUs has markedly decreased the time for reconstruction using inverse scatting approaches. The development of fully 3D image solvers and hardware capable of capturing out of plane scattering have resulted in further improvement in breast tomography. This chapter discusses the state-of-the-art in ultrasound breast tomography, its history, the theory behind inverse scattering, approximations that are included to improve convergence, 3D image reconstruction, and hardware implementation of the constructions.

Review of the Imaging Performance and the Current Status of the Cascade Gamma-Rays Coincidence Imagers

Various studies that have investigated the detection of gamma coincidence events have revealed that design factors and image reconstruction approaches dictate the spatial resolution, coincidence efficiency, and levels of statistical noise of the detection system. In the case of imaging, cascade gamma-ray coincidence (CGC) imagers coupled with collimated detectors offer promising values for both spatial resolution and coincidence efficiency. However, to date, no CGC imager with single or multiple collimated detectors has reported a performance level beyond 6.7 mm spatial resolution (FWHM) and 6.0 ×1ncidence efficiency. Given the recent developments and the current interests in high resolution and localization of an individual decaying nucleus, there is a need for CGC imagers with higher performance in terms of spatial resolution and efficiency. Therefore, deploying a CGC imager coupled with multiple collimated detectors may prove to be of value in nuclear imaging and probably in clinical applications

Superiorization-inspired unrolled SART algorithm with U-Net generated perturbations for sparse-view and limited-angle CT reconstruction

Objective. Unrolled algorithms are a promising approach for reconstruction of CT images in challenging scenarios, such as low-dose, sparse-view and limited-angle imaging. In an unrolled algorithm, a fixed number of iterations of a reconstruction method are unrolled into multiple layers of a neural network, and interspersed with trainable layers. The entire network is then trained end-to-end in a supervised fashion, to learn an appropriate regularizer from training data. In this paper we propose a novel unrolled algorithm, and compare its performance with several other approaches on sparse-view and limited-angle CT. Approach. The proposed algorithm is inspired by the superiorization methodology, an optimization heuristic in which iterates of a feasibility-seeking method are perturbed between iterations, typically using descent directions of a model-based penalty function. Our algorithm instead uses a modified U-net architecture to introduce the perturbations, allowing a network to learn beneficial perturbations to the image at various stages of the reconstruction, based on the training data. Main Results. In several numerical experiments modeling sparse-view and limited angle CT scenarios, the algorithm provides excellent results. In particular, it outperforms several competing unrolled methods in limited-angle scenarios, while providing comparable or better performance on sparse-view scenarios. Significance. This work represents a first step towards exploiting the power of deep learning within the superiorization methodology. Additionally, it studies the effect of network architecture on the performance of unrolled methods, as well as the effectiveness of the unrolled approach on both limited-angle CT, where previous studies have primarily focused on the sparse-view and low-dose cases.

Truly reproducible uniform estimation of the ADC withmulti-b diffusion data- Application in prostate diffusion imaging.

The ADC is a well-established parameter for clinical diagnostic applications, but lacks reproducibility because it is also influenced by the choice diffusion weighting level. A framework is evaluated that is based on multi-b measurement over a wider range of diffusion-weighting levels and higher order tissue diffusion modeling with retrospective, fully reproducible ADC calculation. Averaging effect from curve fitting for various model functions at 20 linearly spaced b-values was determined by means of simulations and theoretical calculations. Simulation and patient multi-b image data were used to compare the new approach for diffusion-weighted image and ADC map reconstruction with and without Rician bias correction to an active clinical trial protocol probing three non-zero b-values. Averaging effect at a certain b-value varies for model function and maximum b-value used. Images and ADC maps from the novel procedure are on-par with the clinical protocol. Higher order modeling and Rician bias correction is feasible, but comes at the cost of longer computation times. Application of the new framework makes higher order modeling more feasible in a clinical setting while still providing patient images and reproducible ADC maps of adequate quality.

Hybrid method for improving Tikhonov-based reconstruction quality in electrical impedance tomography.

Electrical impedance tomography (EIT) has shown its potential in the field of medical imaging. Physiological or pathological variation would cause the change of conductivity. EIT is favorable in reconstructing conductivity distribution inside the detected area. However, due to its ill-posed and nonlinear characteristics, reconstructed images suffer from low spatial resolution. Tikhonov regularization method is a popular and effective approach for image reconstruction in EIT. Nevertheless, excessive smoothness is observed when reconstruction is conducted based on Tikhonov method. To improve Tikhonov-based reconstruction quality in EIT, an innovative hybrid iterative optimization method is proposed. An efficient alternating minimization algorithm is introduced to solve the optimization problem. To verify image reconstruction performance and anti-noise robustness of the proposed method, a series of simulation work and phantom experiments is carried out. Meanwhile, comparison is made with reconstruction results based on Landweber, Newton-Raphson, and Tikhonov methods. The reconstruction performance is also verified by quantitative comparison of blur radius and structural similarity values which further demonstrates the excellent performance of the proposed method. In contrast to Landweber, Newton-Raphson, and Tikhonov methods, it is found that images reconstructed by the proposed method are more accurate. Even under the impact of noise, the proposed method outperforms comparison methods.

Deep Learning--Based Dictionary Learning and Tomographic Image Reconstruction

This work presents an approach for image reconstruction in clinical low-dose tomography that combines principles from sparse signal processing with ideas from deep learning. First, we describe sparse signal representation in terms of dictionaries from a statistical perspective and interpret dictionary learning as a process of aligning distribution that arises from a generative model with empirical distribution of true signals. As a result we can see that sparse coding with learned dictionaries resembles a specific variational autoencoder, where the decoder is a linear function and the encoder is a sparse coding algorithm. Next, we show that dictionary learning can also benefit from computational advancements introduced in the context of deep learning, such as parallelism and as stochastic optimization. Finally, we show that regularization by dictionaries achieves competitive performance in computed tomography (CT) reconstruction comparing to state-of-the-art model based and data driven approaches.

Characteristic-constrained accelerating MR T1rho mapping with blockwise infimal convolution of matrix elastic-net regularization.

Magnetic resonance parameter mapping (MRPM) plays an important role in clinical applications and biomedical researches. However, the acceleration of MRPM remains a major challenge for achieving furtherimprovements. In this work, a new undersampled k-space based joint multi-contrast image reconstruction approach named CC-IC-LMEN is proposed for accelerating MR T1rhomapping. The reconstruction formulation of the proposed CC-IC-LMEN method imposes a blockwise low-rank assumption on the characteristic-image series (c-p space) and utilizes infimal convolution (IC) to exploit and balance the generalized low-rank properties in low-and high-order c-p spaces, thereby improving the accuracy. In addition, matrix elastic-net (MEN) regularization based on the nuclear and Frobenius norms is incorporated to obtain stable and exact solutions in cases with large accelerations and noisy observations. This formulation results in a minimization problem, that can be effectively solved using a numerical algorithm based on the alternating direction method of multipliers (ADMM). Finally, T1rho maps are then generated according to the reconstructed images using nonlinear least-squares (NLSQ) curve fitting with an established relaxometrymodel. The relative l2 -norm error (RLNE) and structural similarity (SSIM) in the regions of interest (ROI) show that the CC-IC-LMEN approach is more accurate than other competing methods even in situations with heavy undersampling or noisyobservation. Our proposed CC-IC-LMEN method provides accurate and robust solutions for accelerated MR T1rhomapping.

DuDoSS: Deep-learning-based dual-domain sinogram synthesis from sparsely sampled projections of cardiac SPECT.

Myocardial perfusion imaging (MPI) using single-photon emission-computed tomography (SPECT) is widely applied for the diagnosis of cardiovascular diseases. In clinical practice, the long scanning procedures and acquisition time might induce patient anxiety and discomfort, motion artifacts, and misalignments between SPECT and computed tomography (CT). Reducing the number of projection angles provides a solution that results in a shorter scanning time. However, fewer projection angles might cause lower reconstruction accuracy, higher noise level, and reconstruction artifacts due to reduced angular sampling. We developed a deep-learning-based approach for high-quality SPECT image reconstruction using sparsely sampled projections. We proposed a novel deep-learning-based dual-domain sinogram synthesis (DuDoSS) method to recover full-view projections from sparsely sampled projections of cardiac SPECT. DuDoSS utilized the SPECT images predicted in the image domain as guidance to generate synthetic full-view projections in the sinogram domain. The synthetic projections were then reconstructed into non-attenuation-corrected and attenuation-corrected (AC) SPECT images for voxel-wise and segment-wise quantitative evaluations in terms of normalized mean square error (NMSE) and absolute percent error (APE). Previous deep-learning-based approaches, including direct sinogram generation (Direct Sino2Sino) and direct image prediction (Direct Img2Img), were tested in this study for comparison. The dataset used in this study included a total of 500 anonymized clinical stress-state MPI studies acquired on a GE NM/CT 850 scanner with 60 projection angles following the injection of 99m Tc-tetrofosmin. Our proposed DuDoSS generated more consistent synthetic projections and SPECT images with the ground truth than other approaches. The average voxel-wise NMSE between the synthetic projections by DuDoSS and the ground-truth full-view projections was 2.08%±0.81%, as compared to 2.21%±0.86% (p<0.001) by Direct Sino2Sino. The averaged voxel-wise NMSE between the AC SPECT images by DuDoSS and the ground-truth AC SPECT images was 1.63%±0.72%, as compared to 1.84%±0.79% (p<0.001) by Direct Sino2Sino and 1.90%±0.66% (p<0.001) by Direct Img2Img. The averaged segment-wise APE between the AC SPECT images by DuDoSS and the ground-truth AC SPECT images was 3.87%±3.23%, as compared to 3.95%±3.21% (p=0.023) by Direct Img2Img and 4.46%±3.58% (p<0.001) by Direct Sino2Sino. Our proposed DuDoSS is feasible to generate accurate synthetic full-view projections from sparsely sampled projections for cardiac SPECT. The synthetic projections and reconstructed SPECT images generated from DuDoSS are more consistent with the ground-truth full-view projections and SPECT images than other approaches. DuDoSS can potentially enable fast data acquisition of cardiac SPECT.

Unsupervised knowledge-transfer for learned image reconstruction* *The work of RB is substantially supported by the i4health PhD studentship (UK EPSRC EP/S021930/1) and from The Alan Turing Institute (UK EPSRC EP/N510129/1), and that of ZK, SA and BJ by UK EPSRC EP/T000864/1, and that of SA and BJ also by UK EPSRC EP/V026259/1. AH acknowledges funding by Academy of Finland Projects

Deep learning-based image reconstruction approaches have demonstrated impressive empirical performance in many imaging modalities. These approaches usually require a large amount of high-quality paired training data, which is often not available in medical imaging. To circumvent this issue we develop a novel unsupervised knowledge-transfer paradigm for learned reconstruction within a Bayesian framework. The proposed approach learns a reconstruction network in two phases. The first phase trains a reconstruction network with a set of ordered pairs comprising of ground truth images of ellipses and the corresponding simulated measurement data. The second phase fine-tunes the pretrained network to more realistic measurement data without supervision. By construction, the framework is capable of delivering predictive uncertainty information over the reconstructed image. We present extensive experimental results on low-dose and sparse-view computed tomography showing that the approach is competitive with several state-of-the-art supervised and unsupervised reconstruction techniques. Moreover, for test data distributed differently from the training data, the proposed framework can significantly improve reconstruction quality not only visually, but also quantitatively in terms of PSNR and SSIM, when compared with learned methods trained on the synthetic dataset only.

Critique of “MemXCT: Memory-Centric X-Ray CT Reconstruction With Massive Parallelization” by SCC Team From Georgia Tech

For the 2020 Student Cluster Competition, we reproduced results from “MemXCT: Memory-Centric X-ray CT Reconstruction with Massive Parallelization” (Hidayetoğlu <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> ). Reproducibility is of critical importance to the scientific community, not just to verify correctness of results but also to see how easily others can understand and work with the given methods. MemXCT is an approach for image reconstruction in X-ray ptychography, which has a broad range of applications in materials science. MemXCT is not the only X-ray tomography algorithm, though; as opposed to compute-centric algorithms, it is designed to scale better by optimizing for memory bandwidth and memory latency. MemXCT also applies several key optimizations in order to ease memory pressure. In this article, we test the performance and strong scaling of MemXCT on 1 to 256 AMD CPU cores (1-4 nodes) and 1-16 Nvidia V100 GPUs (1-4 nodes). We confirm the impact of MemXCT’s optimizations. Still, we find that the performance of some important loops in the MemXCT kernel is much lower on the AMD processors (with AVX2) of our CPU nodes compared to the Intel CPUs (with AVX-512) used in the original article. We also confirm MemXCT performance on Tesla V100 GPUs, as reported in the article.

Temporal Information-Guided Generative Adversarial Networks for Stimuli Image Reconstruction From Human Brain Activities

Understanding how the human brain works has attracted increasing attention in both fields of neuroscience and machine learning. Previous studies use autoencoder and generative adversarial networks (GANs) to improve the quality of stimuli image reconstruction from functional magnetic resonance imaging (fMRI) data. However, these methods mainly focus on acquiring relevant features between two different modalities of data, i.e., stimuli images and fMRI, while ignoring the temporal information of fMRI data, thus leading to suboptimal performance. To address this issue, in this article, we propose a temporal information-guided GAN (TIGAN) to reconstruct visual stimuli from human brain activities. Specifically, the proposed method consists of three key components, including: 1) an image encoder for mapping the stimuli images into latent space; 2) a long short-term memory (LSTM) generator for fMRI feature mapping, which is used to capture temporal information in fMRI data; and 3) a discriminator for image reconstruction, which is used to make the reconstructed image more similar to the original image. In addition, to better measure the relationship of two different modalities of data (i.e., fMRI and natural images), we leverage a pairwise ranking loss to rank the stimuli images and fMRI to ensure strongly associated pairs at the top and weakly related ones at the bottom. The experimental results on real-world data sets suggest that the proposed TIGAN achieves better performance in comparison with several state-of-the-art image reconstruction approaches.

Multistage adaptive noise reduction technique for optical resolution photoacoustic microscopy.

Photoacoustic microscopy has received great attention due to the benefits of the optical resolution contrast as well as its superior spatial resolution and relatively deep depth. Like other imaging modalities, photoacoustic images suffer from noise, and filtering techniques are required to remove them. To overcome the noise, we proposed a combination of filters, including an adaptive median filter, an effective filter for impulsive noise, and a nonlocal means filter, an effective filter for background noise, for noise removal and image quality enhancement. Our proposed method enhanced the signal-to-noise ratio by 16 dB in an in vivo study compared to the traditional image reconstruction approach and preserved the image detail with minimal blurring, which usually occurs when filtering. These experimental results verified that the proposed adaptive multistage denoising techniques could effectively improve image quality under noisy data acquisition conditions, providing a strong foundation for photoacoustic microscopy with limited laser power.

A globally convergent derivative-free projection algorithm for signal processing

In this paper, a hybrid derivative-free conjugate gradient method that inherits the structures of two conjugate gradient methods is introduced to recover sparse signal in compressive sensing by solving the nonlinear convex constrained equations. The global convergence of the proposed method is proved, under some appropriate assumptions. Numerical experiments and comparisons suggest that the proposed algorithm is an efficient approach for sparse signal and image reconstruction in compressive sensing.

Structural alterations in cell organelles induced by photodynamic treatment with chlorin p6 -histamine conjugate in human oral carcinoma cells probed by 3D fluorescence microscopy.

We have explored the intracellular cell organelle's structural alterations after photodynamic treatment with chlorin p6 -histamine conjugate (Cp6 -his) in human oral cancer cells. Herein, the cells were treated with Cp6 -his (10μm) and counterstained with organelle-specific fluorescence probes to find the site of intracellular localization using confocal microscopy. For photodynamic therapy (PDT), the cells were exposed to ~30 kJ/m2 red light (660 ± 20 nm) to induce ~90% cytotoxicity. We used the three-dimensional (3D) image reconstruction approach to analyze the photodynamic damage to cell organelles. The result showed that Cp6 -his localized mainly in the endoplasmic reticulum (ER) and lysosomes but not in mitochondria and Golgi apparatus (GA). The 3D model revealed that in necrotic cells, PDT led to extensive fragmentation of ER and fragmentation and swelling of GA as well. Results suggest that the indirect damage to GA occurred due to loss of connection between ER and GA. Moreover, in damaged cells with no sign of necrosis, the perinuclear ER appeared condensed and surrounded by several small clumps at the peripheral region of the cell, and the GA was observed to form a single condensed structure. Since these structural changes were associated with apoptotic cell death, it is suggested that the necrotic and apoptotic death induced by PDT with Cp6 -his is determined by the severity of damage to ER and indirect damage to GA. The results suggest that the indirect damage to cell organelle apart from the sites of photosensitizer localization and the severity of damage at the organelle level contribute significantly to the mode of cell death in PDT.

Deep Learning-Based Image Reconstruction for Different Medical Imaging Modalities.

Image reconstruction in magnetic resonance imaging (MRI) and computed tomography (CT) is a mathematical process that generates images at many different angles around the patient. Image reconstruction has a fundamental impact on image quality. In recent years, the literature has focused on deep learning and its applications in medical imaging, particularly image reconstruction. Due to the performance of deep learning models in a wide variety of vision applications, a considerable amount of work has recently been carried out using image reconstruction in medical images. MRI and CT appear as the ultimate scientifically appropriate imaging mode for identifying and diagnosing different diseases in this ascension age of technology. This study demonstrates a number of deep learning image reconstruction approaches and a comprehensive review of the most widely used different databases. We also give the challenges and promising future directions for medical image reconstruction.

Learning-based high-quality image recovery from 1D signals obtained by single-pixel imaging

As an innovative and computational imaging technique, it is critical for single-pixel imaging (SPI) to achieve a high reconstruction quality. However, the reconstruction image quality in conventional SPI is heavily dependent on the sampling rate. In this work, we present a learning-based SPI approach for high-quality image reconstruction under a low sampling rate. A bucket detector included in our optical setup is used to collect the one-dimensional (1D) signals reflected from a two-dimensional (2D) object and an end-to-end generative adversarial network (EGAN) which is pre-trained with simulated data is utilized to implement the reconstruction of the object. The results show that the proposed approach is able to produce high quality approximations of 2D images from optically collected 1D bucket signals at a very low sampling ratio. It can also be shown a better performance can be achieved by compared with previous studies on the same dataset, such as conventional SPI, compressive-sensing ghost imaging (CSGI) and U-net-based SPI approaches.

Bayesian Imaging with Data-Driven Priors Encoded by Neural Networks

This paper proposes a new methodology for performing Bayesian inference in imaging inverse problems where the prior knowledge is available in the form of training data. Following the manifold hypothesis, we adopt a data-driven prior that is supported on a submanifold of the ambient space, which we can learn from the training data using a generative model, such as a variational autoencoder or generative adversarial network. We establish the existence and well-posedness of the associated posterior distribution and posterior moments under easily verifiable conditions, providing a rigorous underpinning for Bayesian estimators and uncertainty quantification analyses. Bayesian computation is performed using a parallel tempered version of the pCN algorithm on the manifold, which is shown to be ergodic and robust to the nonconvex nature of these data-driven models. In addition to point estimators and uncertainty quantification analyses, we derive a model misspecification test to automatically detect situations where the data-driven prior is unreliable, and we explain how to identify the dimension of the latent space directly from the training data. The proposed approach is illustrated with a range of experiments with the MNIST dataset and is compared with some variational and message passing image reconstruction approaches from the state of the art that also use data-driven regularization. A model accuracy analysis suggests that the Bayesian probabilities reported by the proposed data-driven models are also accurate under a frequentist definition of probability, suggesting that the learnt prior is close to the true marginal distribution of the unknown image.

Tomographic single pixel spatial frequency projection imaging

Conventional methods for three-dimensional (3D) imaging frequently rely on voxel-by-voxel data acquisition, which restricts the range of specimens in which they can be effectively employed. While advances in imaging technology now permit the routine acquisition of 3D images approaching video rates, there are other limitations to image formation in fluorescent microscopy that prohibit studies in large volume samples, highly scattering media, and dynamic environments. Some approaches to 3D image collection circumvent this need by the use of tomographic imaging, where sub-3D projections are collected at varying illumination angles and reconstructed through an inversion algorithm to compute an estimate of the 3D fluorophore distribution. Many such methods rely on spatially coherent light, and thus prohibit the use of fluorescent light. By employing unique spatio-temporally varying illumination patterns in conjunction with computational imaging approaches to image reconstruction, we show that some limitations of laser scanning and wide-field imaging can be overcome. We outline several approaches that utilize tomographic projections with patterned illumination to collect 3D image data. All three dimensional optical imaging exploits projection of the desired 3D information into a lower-dimensional subspace, and then a full three dimensional object is estimated from these data. We discuss a number of such single pixel strategies that project object information onto a zero-dimensional, usually a power, measurement. Further, we outline computational image reconstruction approaches that enhance the object estimates by employing a forward model for the image formation process.