Nonlinear Image Reconstruction Research Articles

Image quality evaluation in deep-learning-based CT noise reduction using virtual imaging trial methods: Contrast-dependent spatial resolution.

Deep-learning-based image reconstruction and noise reduction methods (DLIR) have been increasingly deployed in clinical CT. Accurate image quality assessment of these methods is challenging as the performance measured using physical phantoms may not represent the true performance of DLIR in patients since DLIR is trained mostly on patient images. In this work, we aim to develop a patient-data-based virtual imaging trial framework and, as a first application, use it to measure the spatial resolution properties of a DLIR method. The patient-data-based virtual imaging trial framework consists of five steps: (1) insertion of lesions into projection domain data using the acquisition geometry of the patient exam to simulate different lesion characteristics; (2) insertion of noise into projection domain data using a realistic photon statistical model of the CT system to simulate different dose levels; (3) creation of DLIR-processed images from projection or image data; (4) creation of ensembles of DLIR-processed patient images from a large number of noise and lesion realizations; and (5) evaluation of image quality using ensemble DLIR images. This framework was applied to measure the spatial resolution of a ResNet based deep convolutional neural network (DCNN) trained on patient images. Lesions in a cylindrical shape and different contrast levels (-500, -100, -50, -20, -10 HU) were inserted to the lower right lobe of the liver in a patient case. Multiple dose levels were simulated (50%, 25%, 12.5%). Each lesion and dose condition had 600 noise realizations. Multiple reconstruction and denoising methods were used on all the noise realizations, including the original filtered-backprojection (FBP), iterative reconstruction (IR), and the DCNN method with three different strength setting (DCNN-weak, DCNN-medium, and DCNN-strong). Mean lesion signal was calculated by performing ensemble averaging of all the noise realizations for each lesion and dose condition and then subtracting the lesion-present images from the lesion absent images. Modulation transfer functions (MTFs) both in-plane and along the z-axis were calculated based on the mean lesion signals. The standard deviations of MTFs at each condition were estimated with bootstrapping: randomly sampling (with replacement) all the DLIR/FBP/IR images from the ensemble data (600 samples) at each condition. The impact of varying lesion contrast, dose levels, and denoising strengths were evaluated. Statistical analysis with paired t-test was used to compare the z-axis and in-plane spatial resolution of five algorithms for five different contrasts and three dose levels. The in-plane and z-axis spatial resolution degradation of DCNN becomes more severe as the contrast or radiation dose decreased, or DCNN denoising strength increased. In comparison with FBP, a 59.5% and 4.1% reduction of in-plane and z-axis MTF (in terms of spatial frequencies at 50% MTF), respectively, was observed at low contrast (-10 HU) for DCNN with the highest denoising strength at 25% routine dose level. When the dose level reduces from 50% to 12.5% of routine dose, the in-plane and z-axis MTFs reduces from 92.1% to 76.3%, and from 98.9% to 95.5%, respectively, at contrast of -100 HU, using FBP as the reference. For most conditions of contrasts and dose levels, significant differences were found among the five algorithms, with the following relationship in both in-plane and cross-plane spatial resolution: FBP>DCNN-Weak>IR>DCNN-Medium>DCNN-Strong. The spatial resolution difference among algorithms decreases at higher contrast or dose levels. A patient-data-based virtual imaging trial framework was developed and applied to measuring the spatial resolution properties of a DCNN noise reduction method at different contrast and dose levels using real patient data. As with other non-linear image reconstruction and post-processing techniques, the evaluated DCNN method degraded the in-plane and z-axis spatial resolution at lower contrast levels, lower radiation dose, and higher denoising strength.

Algorithm Fusion for 3D Ground-Penetrating Radar Imaging with Field Examples

Numerous data processing algorithms are available for ground-penetrating radar (GPR) data processing. However, most of the existing processing algorithms are derived from Fourier theory and assume that the system is linear or that data are stationary, which may oversimplify the case. Some nonlinear algorithms are accessible for improvement but generally are for stationary and deterministic systems. To alleviate the dilemma, this study proposes an algorithm fusion scheme that employs standard linear techniques in conjunction with a newer nonlinear and non-stationary method. The linear techniques include linear filtering, migration, and interpolation. The newer method is mainly for nonlinear filtering and image reconstruction. The results can be demonstrated in a two-dimensional single profile (time–distance section) or a 3D visualization if survey lines fulfill the 3D Nyquist sample intervals requirement. Two controlled experiments were conducted to justify the proposed scheme. Then, a field study including two examples was carried out to demonstrate the feasibility of practical applications. Compared with conventional methods, the proposed algorithm fusion provides better visualization and integrative interpretation for GPR imaging.

Non-Linear Reconstruction for ERT Inverse Problem Based on Homotopy Algorithm

Due to the non-linear and non-convex property, classical ERT image reconstruction algorithms are less effective and less accuracy. In this paper, a non-linear and non-convex image reconstruction algorithm based on homotopy method was proposed. The proposed algorithm converted the ERT inverse problem to a multi-object non-convex optimization which promoted the reconstruction accuracy and avoid the local optimal. Experimental validations were conducted. The optimization process is studied which demonstrate the effectiveness. Moreover, the proposed algorithm is compared with 6 other representative algorithms (linear/non-linear, convex/non-convex) at the condition of different distribution and number of objects. The image quality parameters of the proposed algorithms are studied which show that the homotopy algorithm can provide image reconstruction result with higher quality and better stability than the other conventional algorithms.

A review and experimental evaluation of deep learning methods for MRI reconstruction

Following the success of deep learning in a wide range of applications, neural network-based machine-learning techniques have received significant interest for accelerating magnetic resonance imaging (MRI) acquisition and reconstruction strategies. A number of ideas inspired by deep learning techniques for computer vision and image processing have been successfully applied to nonlinear image reconstruction in the spirit of compressed sensing for accelerated MRI. Given the rapidly growing nature of the field, it is imperative to consolidate and summarize the large number of deep learning methods that have been reported in the literature, to obtain a better understanding of the field in general. This article provides an overview of the recent developments in neural-network based approaches that have been proposed specifically for improving parallel imaging. A general background and introduction to parallel MRI is also given from a classical view of k-space based reconstruction methods. Image domain based techniques that introduce improved regularizers are covered along with k-space based methods which focus on better interpolation strategies using neural networks. While the field is rapidly evolving with plenty of papers published each year, in this review, we attempt to cover broad categories of methods that have shown good performance on publicly available data sets. Limitations and open problems are also discussed and recent efforts for producing open data sets and benchmarks for the community are examined.

A signal detection model for quantifying overregularization in nonlinear image reconstruction.

Many useful image quality metrics for evaluating linear image reconstruction techniques do not apply to or are difficult to interpret for nonlinear image reconstruction. The vast majority of metrics employed for evaluating nonlinear image reconstruction are based on some form of global image fidelity, such as image root mean square error (RMSE). Use of such metrics can lead to overregularization in the sense that they can favor removal of subtle details in the image. To address this shortcoming, we develop an image quality metric based on signal detection that serves as a surrogate to the qualitative loss of fine image details. The metric is demonstrated in the context of a breast CT simulation, where different equal-dose configurations are considered. The configurations differ in the number of projections acquired. Image reconstruction is performed with a nonlinear algorithm based on total variation constrained least-squares (TV-LSQ). The resulting images are studied as a function of three parameters: number of views acquired, total variation constraint value, and number of iterations. The images are evaluated visually, with image RMSE, and with the proposed signal-detection-based metric. The latter uses a small signal, and computes detectability in the sinogram and in the reconstructed image. Loss of signal detectability through the image reconstruction process is taken as a quantitative measure of loss of fine details in the image. Loss of signal detectability is seen to correlate well with the blocky or patchy appearance due to overregularization with TV-LSQ, and this trend runs counter to the image RMSE metric, which tends to favor the over-regularized images. The proposed signal detection-based metric provides an image quality assessment that is complimentary to that of image RMSE. Using the two metrics in concert may yield a useful prescription for determining CT algorithm and configuration parameters when nonlinear image reconstruction is used.

Graph convolutional networks for enhanced resolution 3D Electrical Capacitance Tomography image reconstruction

Three dimensional Electrical Capacitance Tomography (3D ECT) is an inexpensive tool for diagnosing non-conductive components of industrial processes. Although relatively mature, it still requires much work to improve its inverse nature of imaging capability. In particular, high resolution 3D ECT image reconstruction is very time-consuming and computationally heavy, and the best-known 3D ECT image reconstruction techniques have already reached their limits. Thus, there is a strong need to change a direction towards modern computational intelligence solutions. Therefore, this work proposes using graph convolutional networks (GCN) to raise the 3D ECT image quality. Mainly, it takes advantage of GCN’s ability to effectively use specific geometrical relationships hidden in the finite modeling unstructured grids commonly used to build 3D ECT images. These relationships are first encoded by a graph representing an ECT volumetric finite element grid. A GCN is next trained in a graph-to-graph framework with pairs of graphs representing high-quality nonlinear image reconstruction results as input and a simulated phantom as output. As a result, a trained GCN model fed with lower resolution 3D ECT image enhances its quality and spatial resolution. Tomographic image quality and resolution enhancement was evaluated using normalized mean square error and Pearson correlation coefficient, which improved by 35.5% and 3.74%, respectively.

Real-time magnetic resonance imaging in pediatric radiology \u2014 new approach to movement and moving children

The recent development of highly undersampled radial gradient echo sequences in combination with nonlinear inverse image reconstruction now allows for MRI examinations in real time. Image acquisition times as short as 20 ms yield MRI videos with rates of up to 50 frames per second with spin density, T1- and T2-type contrast. The addition of an initial 180° inversion pulse achieves accurate T1 mapping within only 4 s. These technical advances promise specific advantages for studies of infants and young children by eliminating the need for sedation or anesthesia. Our preliminary data demonstrate new diagnostic opportunities ranging from dynamic studies of speech and swallowing processes and body movements to a rapid volumetric assessment of brain cerebrospinal fluid spaces in only few seconds. Real-time MRI of the heart and blood flow can be performed without electrocardiogram gating and under free breathing. The present findings support the idea that real-time MRI will complement existing methods by providing long-awaited diagnostic options for patients in early childhood. Major advantages are the avoidance of sedation or anesthesia and the yet unexplored potential to gain insights into arbitrary body functions.

Comparative Analysis of Multiplicative and Additive Noise Based Automated Regularizations in Non-Linear Diffusion Image Reconstruction

Multiplicative and additive noises are often introduced in image signals during the image acquisition process and result into degradation of image features. The work done by Perona and Malik in 1990 and its modified versions revolutionized the way through which noises or speckles are removed. The Perona-Malik model requires tuning of the regularization parameter to control and prevent staircase artifacts in restored images. The current manual tuning is a challenging and time consuming practice when a long queue of images is registered for processing. Attempt to automate the regularization parameter appeared in Perona-Malik model with self-adjusting shape-defining constant. Although both multiplicative and additive noise based automated regularizations were presented, the paper stayed silent on matters concerning the automation method that fits with speckle reduction. This paper therefore, presents a comparative analysis of additive and multiplicative noise based automated regularizations. Simulation results and paired samples T-tests reveal that the multiplicative noise based automation outperforms the additive noise based automation for small speckle variances. However, the two automation methods do not significantly differ when large speckle variances are assumed.

Investigation of the effect of super-resolution in nonlinear inverse scattering

The idea that a solution to a nonlinear inverse scattering problem (ISP) can contain information about the target on a subwavelength scale and thus allow one to achieve super-resolution (spatial resolution beyond the diffraction limit) has been around since the 1990s. However, a solid mathematical theory of super-resolution in nonlinear image reconstruction is still lacking. In this paper, we investigate the effect of super-resolution in nonlinear ISPs (both analytically and numerically) by analyzing several inverse problems in which the limit of spatial resolution can be defined precisely. The conclusions we obtain are not optimistic. Although it is possible to create examples of exactly solvable models in which account of nonlinearity in the ISP results in additional mathematically independent equations (one such example is shown herein), our results indicate that super-resolution is not achievable in any practical sense. Rather, we find that the linear subspace of possible solutions to a band-limited linearized ISP is transformed into a more general curved manifold due to the effects of nonlinearity. In the one-dimensional problem with realistic interaction that we have considered, the manifold can have a slightly smaller dimensionality that the subspace of solutions to the linearized problem but it does not contract to a point and the effect is practically insignificant.

Single-snapshot X-ray imaging for nonlinear compressive tomosynthesis.

Traditional compressive X-ray tomosynthesis uses sequential illumination to interrogate the object, leading to long scanning time and image distortion due to the object variation. This paper proposes a single-snapshot compressive tomosynthesis imaging approach, where the object is simultaneously illuminated by multiple X-ray emitters equipped with coded apertures. Based on rank, intensity and sparsity prior models, a nonlinear image reconstruction framework is established. The coded aperture patterns are optimized based on uniform sensing criteria. Then, a modified split Bregman algorithm is developed to reconstruct the object from the set of nonlinear compressive measurements. It is shown that the proposed method can be used to reduce the inspection time and achieve robust reconstruction with respect to shape variation or motion of objects.

Compressed sensing MRI using an interpolation-free nonlinear diffusion model.

Constraints in extended neighborhood system demand the use of a large number of interpolations in directionality-guided compressed-sensing nonlinear diffusion MR image reconstruction technique. This limits its practical application in terms of computational complexity. The proposed method aims at multifold improvement in its runtime without compromising the image quality. Conventional approach to extended neighborhood computation requires 108 linear interpolations per pixel for 10 sets of neighborhoods. We propose a neighborhood stretching technique that systematically extends the location of neighboring pixels such that 66% to 100% fewer interpolations are required to compute the gradients along multiple directions. A spatial frequency-based deviation measure is then used to choose the most reliable edges from the set of images generated by diffusion along different directions. The semi-interpolated and interpolation-free diffusion techniques proposed in this paper are compared with the fully interpolated diffusion-based reconstruction by reconstruing multiple multichannel in vivo datasets, undersampled using different sampling patterns at various sampling rates. Results indicate a two- to fivefold increase in reconstruction speed with a potential to generate 1 to 2 dB improvement in peak SNR measure. The proposed method outperforms the state-of-the-art fully interpolated diffusion model and generates high-quality reconstructions for different sampling patterns and acceleration factors with a two- to fivefold increment in reconstruction speed. This makes it the most suitable candidate for edge-preserving penalties used in the compressed sensing MRI reconstruction methods.

Image Super-Resolution Based on CNN Using Multilabel Gene Expression Programming

Current mainstream super-resolution algorithms based on deep learning use a deep convolution neural network (CNN) framework to realize end-to-end learning from low-resolution (LR) image to high-resolution (HR) images, and have achieved good image restoration effects. However, as the number of layers in the network is increased, better results are not necessarily obtained, and there will be problems such as slow training convergence, mismatched sample blocks, and unstable image restoration results. We propose a preclassified deep-learning algorithm (MGEP-SRCNN) using Multilabel Gene Expression Programming (MGEP), which screens out a sample sub-bank with high relevance to the target image before image block extraction, preclassifies samples in a multilabel framework, and then performs nonlinear mapping and image reconstruction. The algorithm is verified through standard images, and better objective image quality is obtained. The restoration effect under different magnification conditions is also better.

Deep-Learning Methods for Parallel Magnetic Resonance Imaging Reconstruction: A Survey of the Current Approaches, Trends, and Issues.

Following the success of deep learning in a wide range of applications, neural network-based machine learning techniques have received interest as a means of accelerating magnetic resonance imaging (MRI). A number of ideas inspired by deep learning techniques from computer vision and image processing have been successfully applied to non-linear image reconstruction in the spirit of compressed sensing for both low dose computed tomography and accelerated MRI. The additional integration of multi-coil information to recover missing k-space lines in the MRI reconstruction process, is still studied less frequently, even though it is the de-facto standard for currently used accelerated MR acquisitions. This manuscript provides an overview of the recent machine learning approaches that have been proposed specifically for improving parallel imaging. A general background introduction to parallel MRI is given that is structured around the classical view of image space and k-space based methods. Both linear and non-linear methods are covered, followed by a discussion of recent efforts to further improve parallel imaging using machine learning, and specifically using artificial neural networks. Image-domain based techniques that introduce improved regularizers are covered as well as k-space based methods, where the focus is on better interpolation strategies using neural networks. Issues and open problems are discussed as well as recent efforts for producing open datasets and benchmarks for the community.

Nonlinear reconstruction of weak optical diffused images under turbid water

Forward scattering noise may degrade the imaging resolution and diffuse the image in turbid water. The reconstruction of diffused images hidden by forward scattering noise is crucial for underwater imaging. To overcome the limitation of forward scattering for optical imaging in turbid water, a nonlinear image reconstruction technology is proposed in the experiment. We experimentally demonstrated the reconstruction of the diffused images under turbid water via signal seeded incoherent modulation instability (MI) in a nonlinear photorefractive crystal. The reconstructed image with high quality and the minimum resolution of 28.51lp/mm are observed in the experiment. This is the first time, to the best of our knowledge, that a spatial MI effect is used to process underwater weak optical diffused images in the experiment.

Real-time MRI for the dynamic assessment of fundoplication failure in patients with gastroesophageal reflux disease.

To assess the diagnostic potential of dynamic real-time MRI for fundoplication failure in patients with persistent or recurrent GERD-like (gastroesophageal reflux disease) complaints. Twenty-two consecutive patients (male n = 11; female n = 11; median age 59years) with recurrent or persistent GERD-like symptom after fundoplication were enrolled between 2015 and 2017. Median duration of GERD-like symptoms was 21months. Real-time MRI (3 Tesla) was performed at 40ms temporal resolution using undersampled radial fast low-angle shot acquisitions with nonlinear inverse image reconstruction. MRI movies dynamically visualized bolus transit of pineapple juice through the gastroesophageal junction, position of the fundoplication wrap and recurring hernia or reflux during Valsalva maneuver. MRI results were compared to endoscopic findings. Real-time MRI was successfully completed in all patients without adverse events (average examination time 15min). Morphological correlates for GERD-like symptoms were evident in 20 patients (90.1%) with gastric reflux in 19 cases. Nine patients (40.1%) had wrap disruption and recurrent gastric hernia. Wrap migration or telescoping hernia was detected in nine patients (40.1%). One patient presented with continued reflux despite intact fundoplication wrap. Esophageal dysmotility with delayed bolus passage was observed in one case. On endoscopy, gastric hernia or wrap disruption was diagnosed in seven cases, and esophagitis or Barret's metaplasia in nine cases. Real-time MRI is a fast and safe modality for dynamic imaging after fundoplication, without radiation exposure or administration of gadolinium-based contrast media. In a relevant number of cases, real-time MRI reveals correlates for GERD-like symptoms. • Real-time MRI reliably visualizes the gastroesophageal junction after fundoplication surgery. • Patients with recurring GERD-like symptoms have a high rate of morphological failure patterns that can be identified by real-time MRI. • Dynamic assessment of gastroesophageal junction by real-time MRI is a perspective diagnostic tool for detection of fundoplication failure.

A mixed-order nonlinear diffusion compressed sensing MR image reconstruction.

Avoid formation of staircase artifacts in nonlinear diffusion-based MR image reconstruction without compromising computational speed. Whereas second-order diffusion encourages the evolution of pixel neighborhood with uniform intensities, fourth-order diffusion considers smooth region to be not necessarily a uniform intensity region but also a planar region. Therefore, a controlled application of fourth-order diffusivity function is used to encourage second-order diffusion to reconstruct the smooth regions of the image as a plane rather than a group of blocks, while not being strong enough to introduce the undesirable speckle effect. Proposed method is compared with second- and fourth-order nonlinear diffusion reconstruction, total variation (TV), total generalized variation, and higher degree TV using in vivo data sets for different undersampling levels with application to dictionary learning-based reconstruction. It is observed that the proposed technique preserves sharp boundaries in the image while preventing the formation of staircase artifacts in the regions of smoothly varying pixel intensities. It also shows reduced error measures compared with second-order nonlinear diffusion reconstruction or TV and converges faster than TV-based methods. Because nonlinear diffusion is known to be an effective alternative to TV for edge-preserving reconstruction, the crucial aspect of staircase artifact removal is addressed. Reconstruction is found to be stable for the experimentally determined range of fourth-order regularization parameter, and therefore not does not introduce a parameter search. Hence, the computational simplicity of second-order diffusion is retained.

A New Divide-and-Conquer Method for 3-D Electrical Impedance Tomography

Electrical impedance tomography (EIT) is a patient-safe approach for imaging applications and is a promising technology for biomedical imaging. By applying a small electrical current to living tissue and measuring the electrical potential at different points of the tissue’s boundary, it is possible to solve the inverse problem and to generate a map of the tissue’s conductivities. Although it is a promising method, the reconstruction process remains a complicated task, and real-time nonlinear image reconstruction algorithms have not yet been widely adopted. Recent studies have proposed the use of artificial neural networks (ANN) to solve the EIT inverse problem. ANNs can give a nonlinear conductivity distribution. Although several studies have examined 2-D finite-element models, very little work has been done on 3-D problems. In this paper, a solution based on the divide-and-conquer method and ANNs is used to solve the nonlinear inverse problem without any linearization. The solution presented here reduces the difficulty of training large ANNs, which are commonly required to solve a 3-D EIT problem with artificial intelligence algorithms.

A scaled gradient method for digital tomographic image reconstruction

Digital tomographic image reconstruction uses multiple x-ray projections obtained along a range of different incident angles to reconstruct a 3D representation of an object. For example, computed tomography (CT) generally refers to the situation when a full set of angles are used (e.g., 360 degrees) while tomosynthesis refers to the case when only a limited (e.g., 30 degrees) angular range is used. In either case, most existing reconstruction algorithms assume that the x-ray source is monoenergetic. This results in a simplified linear forward model, which is easy to solve but can result in artifacts in the reconstructed images. It has been shown that these artifacts can be reduced by using a more accurate polyenergetic assumption for the x-ray source, but the polyenergetic model requires solving a large-scale nonlinear inverse problem. In addition to reducing artifacts, a full polyenergetic model can be used to extract additional information about the materials of the object; that is, to provide a mechanism for quantitative imaging. In this paper, we develop an approach to solve the nonlinear image reconstruction problem by incorporating total variation (TV) regularization. The corresponding optimization problem is then solved by using a scaled gradient descent method. The proposed algorithm is based on KKT conditions and Nesterov's acceleration strategy. Experimental results on reconstructed polyenergetic image data illustrate the effectiveness of this proposed approach.

A Plug-and-Play Priors Approach for Solving Nonlinear Imaging Inverse Problems

In the past two decades, nonlinear image reconstruction methods have led to substantial improvements in the capabilities of numerous imaging systems. Such methods are traditionally formulated as optimization problems that are solved iteratively by simultaneously enforcing data consistency and incorporating prior models. Recently, the Plug-and-Play Priors (PPP) framework suggested that by using more sophisticated denoisers, not necessarily corresponding to an optimization objective, it is possible to improve the quality of reconstructed images. In this letter, we show that the PPP approach is applicable beyond linear inverse problems. In particular, we develop the fast iterative shrinkage/thresholding algorithm variant of PPP for model-based nonlinear inverse scattering. The key advantage of the proposed formulation over the original ADMM-based one is that it does not need to perform an inversion on the forward model. We show that the proposed method produces high quality images using both simulated and experimentally measured data.

Light sheet luminescence imaging with Cherenkov excitation in thick scattering media.

Light scattering leads to a severe loss of axial and transverse resolution with depth into tissue, limiting accuracy and value of biomedical luminescence imaging techniques. High-resolution imaging beyond a few-millimeter depth is prohibited because diffusive transport dominates beyond a few scattering distances. In this study, light sheet imaging through scattering media is demonstrated using a radiotherapy linear accelerator to deliver well-defined thin scanned sheets of x-rays. These sheets produce Cherenkov light within the medium, which in turn excites luminescence of an optical probe across the sheet plane. This luminescence can then be imaged by an intensified camera positioned perpendicular to the sheet plane. The precise knowledge of the light sheet position within the medium allowed for efficient attenuation correction of the signal with depth as well as spatial deconvolution of the excitation light. Together these methods allowed for the first time, to the best of our knowledge, high-resolution imaging of tissue-equivalent phantoms up to 3 cm thick, yielding the precise position and shape of luminescent lesions located deep in tissue without the need for nonlinear image reconstruction.