Image Reconstruction Problem Research Articles

Patient-specific hyperparameter learning for optimization-based CT image reconstruction

We propose a hyperparameter learning framework that learns patient-specific hyperparameters for optimization-based image reconstruction problems for x-ray CT applications. The framework consists of two functional modules: (1) a hyperparameter learning module parameterized by a convolutional neural network, (2) an image reconstruction module that takes as inputs both the noisy sinogram and the hyperparameters from (1) and generates the reconstructed images. As a proof-of-concept study, in this work we focus on a subclass of optimization-based image reconstruction problems with exactly computable solutions so that the whole network can be trained end-to-end in an efficient manner. Unlike existing hyperparameter learning methods, our proposed framework generates patient-specific hyperparameters from the sinogram of the same patient. Numerical studies demonstrate the effectiveness of our proposed approach compared to bi-level optimization.

Multi GPU parallelization of maximum likelihood expectation maximization method for digital rock tomography data

Digital rock is an emerging area of rock physics, which involves scanning reservoir rocks using X-ray micro computed tomography (XCT) scanners and using it for various petrophysical computations and evaluations. The acquired micro CT projections are used to reconstruct the X-ray attenuation maps of the rock. The image reconstruction problem can be solved by utilization of analytical (such as Feldkamp–Davis–Kress (FDK) algorithm) or iterative methods. Analytical schemes are typically computationally more efficient and hence preferred for large datasets such as digital rocks. Iterative schemes like maximum likelihood expectation maximization (MLEM) are known to generate accurate image representation over analytical scheme in limited data (and/or noisy) situations, however iterative schemes are computationally expensive. In this work, we have parallelized the forward and inverse operators used in the MLEM algorithm on multiple graphics processing units (multi-GPU) platforms. The multi-GPU implementation involves dividing the rock volumes and detector geometry into smaller modules (along with overlap regions). Each of the module was passed onto different GPU to enable computation of forward and inverse operations. We observed an acceleration of sim 30 times using our multi-GPU approach compared to the multi-core CPU implementation. Further multi-GPU based MLEM obtained superior reconstruction compared to traditional FDK algorithm.

Accurate and robust sparse-view angle CT image reconstruction using deep learning and prior image constrained compressed sensing (DL-PICCS).

Sparse-view CT image reconstruction problems encountered in dynamic CT acquisitions are technically challenging. Recently, many deep learning strategies have been proposed to reconstruct CT images from sparse-view angle acquisitions showing promising results. However, two fundamental problems with these deep learning reconstruction methods remain to be addressed: (1) limited reconstruction accuracy for individual patients and (2) limited generalizability for patient statisticalcohorts. The purpose of this work is to address the previously mentioned challenges in current deep learningmethods. A method that combines a deep learning strategy with prior image constrained compressed sensing (PICCS) was developed to address these two problems. In this method, the sparse-view CT data were reconstructed by the conventional filtered backprojection (FBP) method first, and then processed by the trained deep neural network to eliminate streaking artifacts. The outputs of the deep learning architecture were then used as the needed prior image in PICCS to reconstruct the image. If the noise level from the PICCS reconstruction is not satisfactory, another light duty deep neural network can then be used to reduce noise level. Both extensive numerical simulation data and human subject data have been used to quantitatively and qualitatively assess the performance of the proposed DL-PICCS method in terms of reconstruction accuracy andgeneralizability. Extensive evaluation studies have demonstrated that: (1) quantitative reconstruction accuracy of DL-PICCS for individual patient is improved when it is compared with the deep learning methods and CS-based methods; (2) the false-positive lesion-like structures and false negative missing anatomical structures in the deep learning approaches can be effectively eliminated in the DL-PICCS reconstructed images; and (3) DL-PICCS enables a deep learning scheme to relax its working conditions to enhance itsgeneralizability. DL-PICCS offers a promising opportunity to achieve personalized reconstruction with improved reconstruction accuracy and enhanced generalizability.

A novel computational imaging algorithm based on split Bregman iterative for electrical capacitance tomography

As an advanced detection technology in the industrial field, electrical capacitance tomography (ECT) can better reconstruct the material distribution state in the measured area by selecting the appropriate algorithm. In order to improve the reconstruction quality, this paper devises a novel objective function to model the ECT image reconstruction problem, in which L 1-norm is deployed as data fidelity with the focus on weakening the influence of capacitance outliers on the reconstruction quality, L P regularization reinforces the sparseness of the phantom objects, and L 1 regularization applies to model deviation variables to increase the robustness of the system. Based on the fast-iterative shrinkage thresholding technique and the soft thresholding method as sub-solvers, the split Bregman iterative method is designed as an effective solver for the proposed objective function. Numerical simulation and experimental validate that the proposed algorithm has excellent imaging ability compared with other imaging algorithms, which provides a good choice for the practical application of ECT in the future.

Adjoint image warping using multivariate splines with application to four-dimensional computed tomography.

PurposeAdjoint image warping is an important tool to solve image reconstruction problems that warp the unknown image in the forward model. This includes four‐dimensional computed tomography (4D‐CT) models in which images are compared against recorded projection images of various time frames using image warping as a model of the motion. The inversion of these models requires the adjoint of image warping, which up to now has been substituted by approximations. We introduce an efficient implementation of the exact adjoints of multivariate spline based image warping, and compare it against previously used alternatives.MethodsUsing symbolic computer algebra, we computed a list of 64 polynomials that allow us to compute a matrix representation of trivariate cubic image warping. By combining an on‐the‐fly computation of this matrix with a parallelized implementation of columnwise matrix multiplication, we obtained an efficient, low memory implementation of the adjoint action of 3D cubic image warping. We used this operator in the solution of a previously proposed 4D‐CT reconstruction model in which the image of a single subscan was compared against projection data of multiple subscans by warping and then projecting the image. We compared the properties of our exact adjoint with those of approximate adjoints by warping along inverted motion.ResultsOur method requires halve the memory to store motion between subscans, compared to methods that need to compute and store an approximate inverse of the motion. It also avoids the computation time to invert the motion and the tunable parameter of the number of iterations used to perform this inversion. Yet, a similar and often better reconstruction quality was obtained in comparison with these more expensive methods, especially when the motion is large. When compared against a simpler method that is similar to ours in computational demands, our method achieves a higher reconstruction quality in general.ConclusionsOur implementation of the exact adjoint of cubic image warping improves efficiency and provides accurate reconstructions.

Noise-Robust Image Reconstruction Based on Minimizing Extended Class of Power-Divergence Measures.

The problem of tomographic image reconstruction can be reduced to an optimization problem of finding unknown pixel values subject to minimizing the difference between the measured and forward projections. Iterative image reconstruction algorithms provide significant improvements over transform methods in computed tomography. In this paper, we present an extended class of power-divergence measures (PDMs), which includes a large set of distance and relative entropy measures, and propose an iterative reconstruction algorithm based on the extended PDM (EPDM) as an objective function for the optimization strategy. For this purpose, we introduce a system of nonlinear differential equations whose Lyapunov function is equivalent to the EPDM. Then, we derive an iterative formula by multiplicative discretization of the continuous-time system. Since the parameterized EPDM family includes the Kullback–Leibler divergence, the resulting iterative algorithm is a natural extension of the maximum-likelihood expectation-maximization (MLEM) method. We conducted image reconstruction experiments using noisy projection data and found that the proposed algorithm outperformed MLEM and could reconstruct high-quality images that were robust to measured noise by properly selecting parameters.

Image-based benchmarking and visualization for large-scale global optimization

In the context of optimization, visualization techniques can be useful for understanding the behaviour of optimization algorithms and can even provide a means to facilitate human interaction with an optimizer. Towards this goal, an image-based visualization framework, without dimension reduction, that visualizes the solutions to large-scale global optimization problems as images is proposed. In the proposed framework, the pixels visualize decision variables while the entire image represents the overall solution quality. This framework affords a number of benefits over existing visualization techniques including enhanced scalability (in terms of the number of decision variables), facilitation of standard image processing techniques, providing nearly infinite benchmark cases, and explicit alignment with human perception. To the best of the authors’ knowledge, this is the first realization of a dimension-preserving, scalable visualization framework that embeds the inherent relationship between decision space and objective space. The proposed framework is utilized with different mapping schemes on an image-reconstruction problem that encompass continuous, discrete, constrained, dynamic, and multi-objective optimization. The proposed framework is then demonstrated on arbitrary benchmark problems with known optima. Experimental results elucidate the flexibility and demonstrate how valuable information about the search process can be gathered via the proposed visualization framework. Results of a user survey strongly support that users perceive a correlation between objective fitness values and the quality of the corresponding images generated by the proposed framework.

Perceptual losses for self-supervised depth estimation

Convolution neural network has shown excellent results in stereo and monocular disparity estimation, while most of the existing methods convert the image depth prediction problem into the image reconstruction problem, and calculate the depth of each pixel through the disparity between the generated left and right images. However, in the reconstruction task, the loss is still calculated at the pixel level when comparing the reconstructed picture with the original picture, which will greatly affect the estimation of picture depth due to the problems of illumination and occlusion. Therefore, when calculating the loss of image reconstruction, it is very important to compare the higher-level features extracted from the reconstructed image with the original image. In this paper, based on the existing methods, we have innovated the loss function and introduced perceptual loss, i.e., we use feedforward neural network to extract features to further evaluate the reconstructed image, to make the reconstruction loss of baseline more accurate and improve the accuracy and robustness of the depth prediction model. To compare the improved effect, we performed extensive experiments on KITTI driving data by the improved model set, and the experimental index obtains better performance than the original baseline model.

Numerical inversion and uniqueness of a spherical radon transform restricted with a fixed angular span

In this paper, we study a spherical Radon transform that maps a function to its surface integrals over spheres restricted with a fixed angular span. Such transform is relevant for various image reconstruction problems arising in medical, radar and sonar imaging. This paper contains uniqueness results for the spherical Radon transform in the case of a fixed angular span, valid when the support of the image function is inside or outside the data acquisition sphere. Furthermore, we present simulation results for the numerical inversion in the special case of the spherical cap Radon transform.

Azimuthal jittered sampling of bandlimited functions in the two-dimensional Fourier transform and the Hankel transform domains

Stark’s sampling theorems in polar coordinates have important significance in the field of medical imaging. However, the direct use of Stark’s results may mislead as in most real-world scenarios the jitter occurs in azimuth due to uncertainty of sampling at the medical imaging system transmitter end. On the basis of the classical Stark’s interpolation formulae, this paper addresses the image reconstruction problem when the sample data are collected on nonuniform polar radius and polar azimuth lattices, where the sample points are normalized zeros of Bessel functions in radius while random jittered ones in azimuth that are clustered around uniform points with a given probability distribution. By using the existing basis functions found in Stark’s formulae, it follows a biased estimate of the original angularly periodic images that are bandlimited both to the highest frequency and in the Fourier transform or the Hankel transform domain. With the newly designed angular generating functions, there are a kind of unbiased estimators whose variance approaches zero as the jitter becomes less and less pronounced.

Projection-Based cascaded U-Net model for MR image reconstruction

Background and ObjectiveBackground and Objective: Recent studies in deep learning reveal that the U-Net stands out among the diverse set of deep models as an effective network structure, especially for imaging inverse problems. Initially, the U-Net model was developed to solve segmentation problems for biomedical images while using an annotated dataset. In this paper, we will study a novel application of the U-Net structure for the important inverse problem of MRI reconstruction. Deep networks are particularly efficient for the speed-up of the MR image reconstruction process by decreasing the data acquisition time, and they can significantly reduce the aliasing artifacts caused by the undersampling in the k-space. Our aim is to develop a novel and efficient cascaded U-Net framework for reconstructing MR images from undersampled k-space data. The new framework should have improved reconstruction performance when compared to competing methodologies. MethodsIn this paper, a novel cascaded framework utilizing the U-Net as a sub-block is being proposed. The introduced U-Net cascade structure is applied to the magnetic resonance image reconstruction problem. The connection between the cascaded U-Nets is realized in the form of a recently developed projection-based updated data consistency layer. The novel structure is implemented in the PyTorch environment, which is one of the standards for deep learning implementations. The recently created fastMRI dataset which forms an important benchmark for MRI reconstruction is used for training and testing purposes. ResultsWe present simulation results comparing the novel method with a variety of competitive deep networks. The new cascaded U-Net structures PSNR performance stands on average 1.28 dB higher than the baseline U-Net. The improvement, when compared to the standard CNN, is on average 3.32 dB. ConclusionsThe proposed cascaded U-Net configuration results in an improved reconstruction performance when compared to the CNN, the cascaded CNN, and also the singular U-Net structures, where the singular U-Net forms the baseline reconstruction method from the fastMRI package. The use of the projection-based updated data consistency layer also leads to improved quantitative (including SSIM, PSNR, and NMSE results) and qualitative results when compared to the use of the conventional data consistency layer.

Nonplanar sensing skins for structural health monitoring based on electrical resistance tomography

AbstractElectrical resistance tomography (ERT)‐based distributed surface sensing systems, or sensing skins, offer alternative sensing techniques for structural health monitoring, providing capabilities for distributed sensing of, for example, damage, strain, and temperature. Currently, however, the computational techniques utilized for sensing skins are limited to planar surfaces. In this paper, to overcome this limitation, we generalize the ERT‐based surface sensing to nonplanar surfaces covering arbitrarily shaped three‐dimensional structures; we construct a framework in which we reformulate the image reconstruction problem of ERT using techniques of Riemannian geometry, and solve the resulting problem numerically. We test this framework in series of numerical and experimental studies. The results demonstrate the feasibility of the proposed formulation and the applicability of ERT‐based sensing skins to nonplanar geometries.

Assessing the credibility of the solutions of incomplete-data inverse problems

This paper proposes an approach to measure the credibility of solutions of inverse problems with incomplete or missing data, encountered in some physical problems. In such problems, the same set of data can produce multiple solutions, depending on the constraints or assumptions used to compensate for the missing information. The actual “true” solution, against which the quality of a constrained solution can be measured, is not usually available. In this work, we propose to obtain a complete but coarse reference solution that reliably possesses attributes of the actual solution. This is done by solving an over-complete problem with the same set of data but with a coarser structure. A number of residual, composition and fidelity metrics are then used to quantitatively measure the quality of a solution obtained from incomplete data, against the coarse complete reference. The approach was tested with various degrees of incompleteness and different levels of uncertainty (noise) in data in the linear problem of image reconstruction in computed-tomography. The results show that the approach was successful in identifying credible solutions, as well as those that were not trustworthy.

A strongly convergent Krasnosel’skiǐ–Mann-type algorithm for finding a common fixed point of a countably infinite family of nonexpansive operators in Hilbert spaces

In this article, we propose a Krasnosel’skiǐ–Mann-type algorithm for finding a common fixed point of a countably infinite family of nonexpansive operators (Tn)n≥0 in Hilbert spaces. We formulate an asymptotic property which the family (Tn)n≥0 has to fulfill such that the sequence generated by the algorithm converges strongly to the element in ⋂n≥0FixTn with minimum norm. Based on this, we derive a forward–backward algorithm that allows variable step sizes and generates a sequence of iterates that converge strongly to the zero with minimum norm of the sum of a maximally monotone operator and a cocoercive one. We demonstrate the superiority of the forward–backward algorithm with variable step sizes over the one with constant step size by means of numerical experiments on variational image reconstruction and split feasibility problems in infinite dimensional Hilbert spaces.

An end-to-end-trainable iterative network architecture for accelerated radial multi-coil 2D cine MR image reconstruction.

Iterative convolutional neural networks (CNNs) which resemble unrolled learned iterative schemes have shown to consistently deliver state-of-the-art results for image reconstruction problems across different imaging modalities. However, because these methods include the forward model in the architecture, their applicability is often restricted to either relatively small reconstruction problems or to problems with operators which are computationally cheap to compute. As a consequence, they have not been applied to dynamic non-Cartesian multi-coil reconstruction problems so far. In this work, we propose a CNN architecture for image reconstruction of accelerated 2D radial cine MRI with multiple receiver coils. The network is based on a computationally light CNN component and a subsequent conjugate gradient (CG) method which can be jointly trained end-to-end using an efficient training strategy. We investigate the proposed training strategy and compare our method with other well-known reconstruction techniques with learned and non-learned regularization methods. Our proposed method outperforms all other methods based on non-learned regularization. Further, it performs similar or better than a CNN-based method employing a 3D U-Net and a method using adaptive dictionary learning. In addition, we empirically demonstrate that even by training the network with only iteration, it is possible to increase the length of the network at test time and further improve the results. End-to-end training allows to highly reduce the number of trainable parameters of and stabilize the reconstruction network. Further, because it is possible to change the length of the network at the test time, the need to find a compromise between the complexity of the CNN-block and the number of iterations in each CG-block becomes irrelevant.

First application of the GPU-based software framework TIGRE for proton CT image reconstruction.

In proton therapy, the knowledge of the proton stopping power, i.e. the energy deposition per unit length within human tissue, is essential for accurate treatment planning. One suitable method to directly measure the stopping power is proton computed tomography (pCT). Due to the proton interaction mechanisms in matter, pCT image reconstruction faces some challenges: the unique path of each proton has to be considered separately in the reconstruction process adding complexity to the reconstruction problem. This study shows that the GPU-based open-source software toolkit TIGRE, which was initially intended for X-ray CT reconstruction, can be applied to the pCT image reconstruction problem using a straight line approach for the proton path. This simplified approach allows for reconstructions within seconds. To validate the applicability of TIGRE to pCT, several Monte Carlo simulations modeling a pCT setup with two Catphan® modules as phantoms were performed. Ordered-Subset Simultaneous Algebraic Reconstruction Technique (OS-SART) and Adaptive-Steepest-Descent Projection Onto Convex Sets (ASD-POCS) were used for image reconstruction. Since the accuracy of the approach is limited by the straight line approximation of the proton path, requirements for further improvement of TIGRE for pCT are addressed.

On maximum residual block and two-step Gauss–Seidel algorithms for linear least-squares problems

The block Gauss–Seidel algorithm can significantly outperform the simple randomized Gauss–Seidel algorithm for solving overdetermined least-squares problems since it moves a large block of columns rather than a single column into working memory. Here, with the help of the maximum residual rule, we construct a two-step Gauss–Seidel (2SGS) algorithm, which selects two different columns simultaneously at each iteration. As a natural extension of the 2SGS algorithm, we further propose a multi-step Gauss–Seidel algorithm, that is, the maximum residual block Gauss–Seidel (MRBGS) algorithm for solving overdetermined least-squares problems. We prove that these two different algorithms converge to the unique solution of the overdetermined least-squares problem when its coefficient matrix is of full column rank. Numerical experiments on Gaussian models as well as on 2D image reconstruction problems, show that 2SGS is more effective than the greedy randomized Gauss–Seidel algorithm, and MRBGS apparently outperforms both the greedy and randomized block Gauss–Seidel algorithms in terms of numerical performance.

Ultrasound image reconstruction from plane wave radio-frequency data by self-supervised deep neural network.

Image reconstruction from radio-frequency (RF) data is crucial for ultrafast plane wave ultrasound (PWUS) imaging. Compared with the traditional delay-and-sum (DAS) method based on relatively imprecise assumptions, sparse regularization (SR) method directly solves the inverse problem of image reconstruction and has presented significant improvement in the image quality when the frame rate remains high. However, the computational complexity of SR is too high for practical implementation, which is inherently associated with its iterative process. In this work, a deep neural network (DNN), which is trained with an incorporated loss function including sparse regularization terms, is proposed to reconstruct PWUS images from RF data with significantly reduced computational time. It is remarkable that, a self-supervised learning scheme, in which the RF data are utilized as both the inputs and the labels during the training process, is employed to overcome the lack of the "ideal" ultrasound images as the labels for DNN. In addition, it has been also verified that the trained network can be used on the RF data obtained with steered plane waves (PWs), and thus the image quality can be further improved with coherent compounding. Using simulation data, the proposed method has significantly shorter reconstruction time (∼10ms) than the conventional SR method (∼1-5mins), with comparable spatial resolution and 1.5-dB higher contrast-to-noise ratio (CNR). Besides, the proposed method with single PW can achieve higher CNR than DAS with 75PWs in reconstruction of in-vivo images of human carotid arteries.

Reconstruction of Partially Occluded Facial Image for Classification

The recent literature indicates that face recognition and facial expression classification has achieved a high accuracy on benchmark datasets with a large number of face images in the wild. On the other hand, unlike the purpose of recognizing as many people as possible, real applications for families or companies usually aim to recognize a small group of people as accurate as possible. In case of the face is partially occluded, convolutional solutions always simply put images with occlusions into the training dataset and hope the convolution neural network learn a model robust to partial occlusion. These processes not only increase the burden of learning, but also affect the model to identify normal images without occlusions. In this paper, we investigate the research problem of facial image reconstruction to discard the influence of partial occlusion. Based on the phenomenon that human faces are roughly symmetrical, we propose to reconstruct the facial information of occluded areas with the intact half face. Specifically, occlusion on the left‐half face is reconstructed with a linear combination of features on the right‐half face and vice versa. The process is modeled by keeping row sparsity for the coefficient matrix with l2,1‐norm regularization while minimizing the reconstruction error. An alternative iterative algorithm is proposed to solve the optimization problem. To validate the effectiveness of the reconstruction, we fine‐tune a pretrained model, AlexNet, training on normal face images and test with various occluded images. Extensive experimental results show that our method has improved the classification performance effectively. © 2021 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

A new solution to the curved Ewald sphere problem for 3D image reconstruction in electron microscopy

We develop an algorithm capable of imaging a three-dimensional object given a collection of two-dimensional images of that object that are significantly influenced by the curvature of the Ewald sphere. These two-dimensional images cannot be approximated as projections of the object. Such an algorithm is useful in cryo-electron microscopy where larger samples, higher resolution, or lower energy electron beams are desired, all of which contribute to the significance of Ewald curvature.