Iterative Regularization Algorithm Research Articles

Iterative Regularization with k-support Norm: An Important Complement to Sparse Recovery

Sparse recovery is ubiquitous in machine learning and signal processing. Due to the NP-hard nature of sparse recovery, existing methods are known to suffer either from restrictive (or even unknown) applicability conditions, or high computational cost. Recently, iterative regularization methods have emerged as a promising fast approach because they can achieve sparse recovery in one pass through early stopping, rather than the tedious grid-search used in the traditional methods. However, most of those iterative methods are based on the l1 norm which requires restrictive applicability conditions and could fail in many cases. Therefore, achieving sparse recovery with iterative regularization methods under a wider range of conditions has yet to be further explored. To address this issue, we propose a novel iterative regularization algorithm, IRKSN, based on the k-support norm regularizer rather than the l1 norm. We provide conditions for sparse recovery with IRKSN, and compare them with traditional conditions for recovery with l1 norm regularizers. Additionally, we give an early stopping bound on the model error of IRKSN with explicit constants, achieving the standard linear rate for sparse recovery. Finally, we illustrate the applicability of our algorithm on several experiments, including a support recovery experiment with a correlated design matrix.

An efficient plug-and-play regularization method for full waveform inversion

Abstract Nonlinear inverse problems arise in various fields ranging from scientific computation to engineering technology. Inverse problems are intrinsically ill-posed, and effective regularization techniques are necessary. The core of a suitable regularization method is to introduce the prior information of the model via an explicit or implicit regularization function. Plug-and-play regularization is a flexible framework that integrates the most effective denoising priors into an iterative algorithm, and it has recently shown great potential in the solution of linear ill-posed problems. Unlike traditional regularization methods, plug-and-play regularization does not require an explicit regularization function to represent the prior information of the model. In this work, by using total variation, block-matching and three-dimensional filtering, and fast and flexible denoising convolutional neural network denoisers, we propose a novel iterative regularization algorithm based on the alternating direction method of multipliers method. The combination of total variation and block-matching three-dimensional filtering regularizers can take advantage of the sparsity and nonlocal similarity in the solution of inverse problems. When combined with traditional and novel regularizers, deep neural networks have been shown to be an effective regularization approach, which can achieve state-of-the-art performance. Finally, we apply the proposed algorithm to the full waveform inversion problem to show the effectiveness of our method. Numerical results demonstrate that the proposed algorithm outperforms existing inversion methods in terms of quantitative measures and visual perceptual quality.

A CCBM-based generalized GKB iterative regularization algorithm for inverse Cauchy problems

This paper examines inverse Cauchy problems that are governed by a kind of elliptic partial differential equation. The inverse problems involve recovering the missing data on an inaccessible boundary from the measured data on an accessible boundary, which is severely ill-posed. By using the coupled complex boundary method (CCBM), which integrates both Dirichlet and Neumann data into a single Robin boundary condition, we reformulate the underlying problem into an operator equation. Based on this new formulation, we study the solution existence issue of the reduced problem with noisy data. A Golub–Kahan bidiagonalization (GKB) process together with Givens rotation is employed for iteratively solving the proposed operator equation. The regularizing property of the developed method, called CCBM-GKB, and its convergence rate results are proved under a posteriori stopping rule. Finally, a linear finite element method is used for the numerical realization of CCBM-GKB. Various numerical experiments demonstrate that CCBM-GKB is a kind of accelerated iterative regularization method, as it is much faster than the classic Landweber method.

Robust reconstruction of fluorescence molecular tomography based on adaptive adversarial learning strategy

Objective. Fluorescence molecular tomography (FMT) is a promising molecular imaging modality for quantifying the three-dimensional (3D) distribution of tumor probes in small animals. However, traditional deep learning reconstruction methods that aim to minimize the mean squared error (MSE) and iterative regularization algorithms that rely on optimal parameters are typically influenced by strong noise, resulting in poor FMT reconstruction robustness. Approach. In this letter, we propose an adaptive adversarial learning strategy (3D-UR-WGAN) to achieve robust FMT reconstructions. Unlike the pixel-based MSE criterion in traditional CNNs or the regularization strategy in iterative solving schemes, the reconstruction strategy can greatly facilitate the performance of the network models through alternating loop training of the generator and the discriminator. Second, the reconstruction strategy combines the adversarial loss in GANs with the L1 loss to significantly enhance the robustness and preserve image details and textual information. Main results. Both numerical simulations and physical phantom experiments demonstrate that the 3D-UR-WGAN method can adaptively eliminate the effects of different noise levels on the reconstruction results, resulting in robust reconstructed images with reduced artifacts and enhanced image contrast. Compared with the state-of-the-art methods, the proposed method achieves better reconstruction performance in terms of target shape recovery and localization accuracy. Significance. This adaptive adversarial learning reconstruction strategy can provide a possible paradigm for robust reconstruction in complex environments, and also has great potential to provide an alternative solution for solving the problem of poor robustness encountered in other optical imaging modalities such as diffuse optical tomography, bioluminescence imaging, and Cherenkov luminescence imaging.

Comparative study on the applicability of regularization method in ship’s magnetic signature modeling based on monopole array

In this paper, we first study the applicability and stability of the Tikhonov regularization method (TRM) and the truncated singular value decomposition method in ship’s magnetic signature inversion modeling. To further improve the noise immunity and robustness of the model, we apply an iterative regularization algorithm based on the conjugate gradient least squares (CGLS) method. Numerical simulation shows that the proposed algorithm is more applicable and can obtain better results of ship’s magnetic field extrapolation under different noise backgrounds. In the ship model experiment, the errors of the TRM and CGLS method for magnetic signature extrapolation are, respectively, 4.77% and 3.67%, showing that the proposed algorithm can improve the inversion effect of the ship model magnetic field.

An Iterative Regularization Method Based on Tensor Subspace Representation for Hyperspectral Image Super-Resolution

Hyperspectral image super-resolution (HSI-SR) can be achieved by fusing a paired multispectral image (MSI) and hyperspectral image (HSI), which is a prevalent strategy. But, how to precisely reconstruct the high spatial resolution hyperspectral image (HR-HSI) by fusion technology is a challenging issue. In this paper, we propose an iterative regularization method based on tensor subspace representation (IR-TenSR) for MSI-HSI fusion, thus HSI-SR. First, we propose a tensor subspace representation (TenSR)-based regularization model that integrates the global spectral-spatial low-rank and the nonlocal self-similarity priors of HR-HSI. These two priors have been proven effective, but previous HSI-SR works cannot simultaneously exploit them. Subsequently, we design an iterative regularization procedure to utilize the residual information of acquired low-resolution images, which are ignored in other works that produce suboptimal results. Finally, we develop an effective algorithm based on the proximal alternating minimization method to solve the TenSR-regularization model. With that, we obtain the iterative regularization algorithm. Experiments implemented on the simulated and real datasets illustrate the advantages of the proposed IR-TenSR compared with state-of-the-art fusion approaches. The code is available at https://github.com/liangjiandeng/IR-TenSR.

Discretization of equations Gelfand-Levitan-Krein and regularization algorithms

The paper considers the initial-boundary-value inverse problem of acoustics for onedimensional and multidimensional cases. The inverse problems are to reconstruct the coefficients using one-dimensional and multidimensional analogues of the Gelfand-Levitan-Krein integral equations. It is known that such equations are linear integral Fredholm equations of the first kind, which are ill-posed. The aim of the work is to find a numerical solution of the Gelfand-Levitan-Krein equation using iterative regularizing algorithms. Using the specifics of these equations (the kernel of the equation depends on the difference of arguments) it is possible to create highly efficient iterative regularizing algorithms. The implemented algorithms can be successfully applied in solving such problems as reconstruction of blurred and defocused images, inverse problem of gravimetric, linear programming problem with inaccurately given matrix of constraints, inverse problem of Geophysics, inverse problems of computed tomography, etc. The main results of the work are the discretization of the one-dimensional and multidimensional Gelfand-Levitan-Krein equation and the construction of iterative regularization algorithms.

Iterative Algorithms for Deblurring of Images in Case of Electrical Capacitance Tomography

Electrical capacitance tomography (ECT) has been used to measure flow by applying gas-solid flow in coal gasification, pharmaceutical, and other industries. ECT is also used for creating images of physically confined objects. The data collected by the acquisition system to produce images undergo blurring because of ambient conditions and the electronic circuitry used. This research includes the principle of ECT techniques for deblurring images that were created during measurement. The data recorded by the said acquisition system ascends a large number of linear equations. This system of equations is sparse and ill-conditioned and hence is ill-posed in nature. A variety of reconstruction algorithms with many pros and cons are available to deal with ill-posed problems. Large-scale systems of linear equations resulting during image deblurring problems are solved using iterative regularization algorithms. The conjugate gradient algorithm for least-squares problems (CGLS), least-squares QR factorization (LSQR), and the modified residual norm steepest descent (MRNSD) algorithm are the famous variations of iterative algorithms. These algorithms exhibit a semiconvergence behavior; that is, the computed solution quality first improves and then reduces as the error norm decreases and later increases with each iteration. In this work, soft thresholding has been used for image deblurring problems to tackle the semiconvergence issues. Numerical test problems were executed to indicate the efficacy of the suggested algorithms with criteria for optimal stopping iterations. Results show marginal improvement compared to the traditional iterative algorithms (CGLS, LSQR, and MRNSD) for resolving semiconvergence behavior and image restoration.

Application of Deep Neural Network to the Reconstruction of Two-Phase Material Imaging by Capacitively Coupled Electrical Resistance Tomography

A convolutional neural network (CNN)-based image reconstruction algorithm for two-phase material imaging is presented and verified with experimental data from a capacitively coupled electrical resistance tomography (CCERT) sensor. As a contactless version of electrical resistance tomography (ERT), CCERT has advantages such as no invasion, low cost, no radiation, and rapid response for two-phase material imaging. Besides that, CCERT avoids contact error of ERT by imaging from outside of the pipe. Forward modeling was implemented based on the practical circular array sensor, and the inverse image reconstruction was realized by a CNN-based supervised learning algorithm, as well as the well-known total variation (TV) regularization algorithm for comparison. The 2D, monochrome, 2500-pixel image was divided into 625 clusters, and each cluster was used individually to train its own CNN to solve the 16 classes classification problem. Inherent regularization for the assumption of binary materials enabled us to use a classification algorithm with CNN. The iterative TV regularization algorithm achieved a close state of the two-phase material reconstruction by its sparsity-based assumption. The supervised learning algorithm established the mathematical model that mapped the simulated resistance measurement to the pixel patterns of the clusters. The training process was carried out only using simulated measurement data, but simulated and experimental tests were both conducted to investigate the feasibility of applying a multi-layer CNN for CCERT imaging. The performance of the CNN algorithm on the simulated data is demonstrated, and the comparison between the results created by the TV-based algorithm and the proposed CNN algorithm with the real-world data is also provided.

Iterative regularization methods for solving equilibrium problems

The paper concerns with two new schemes for approximating solutions of equilibrium problems involving monotone and Lipschitz-type bifunctions in Hilbert spaces. We describe how to incorporate the two-step proximal-like method (modified extragradient method) with the regularization technique and then we propose two iterative regularization algorithms for solving equilibrium problems. Unlike the viscosity-like methods, we establish the strong convergence of the new algorithms based on the regularization. The first algorithm is designed to work with the prior knowledge of Lipschitz-type constants of bifunction while the second one, with a simple stepsize rule, is done without this requirement. In order to illustrate the effectiveness and the convergence of the algorithms, we provide several numerical experiments in comparisons with other well known algorithms, which show that our new algorithms are effective for solving equilibrium problems.wi

Optimal Shape of Non-Linear Partially Wet Annular Fins for Maximum Efficiency

Heat exchangers with annular finned-tube type and partially wetted condition are utilized widely in engineering systems, such as air-conditioning systems and refrigeration systems. In addition, the physical properties of fin materials should be considered as functions of temperature in reality and thus become a non-linear problem. Based on the above two conditions, an optimal partially wet annular fin design problem, with temperature-dependent thermal properties of the fin, to yield optimal fin efficiency was investigated in the present work, which has not been examined previously and it is the novelty of this study. An iterative regularization algorithm using the conjugate gradient method (CGM) is considered as the optimization tool based on the desired fin efficiency under a fixed fin volume constraint. The partially wet annular fin condition can result if the relative humidity of surrounding air is between 80 and 90%. Finally, the optimal fin shape, with the highest computed efficiency among examined fins under identical operational conditions, can be obtained. It is found that when the Biot numbers for ambient air (Bia) and relative humidity (φ) increased, the optimum computed fin efficiency and interfacial radius between wet and dry fin domains (rwd) will be increased, and the estimated optimum fin shape also changed. However, the shape of optimal fin remained approximately unchanged when the Biot numbers for the inner tube (Bii), the thermal conductivities of the tube (kw) and fin (kf) varied. It reveals that Bii, kw and kf have an insignificant influence on the optimal shape of the annular fin in a partially wet condition.

Two new non-negativity preserving iterative regularization methods for ill-posed inverse problems

Many inverse problems are concerned with the estimation of non-negative parameter functions. In this paper, in order to obtain non-negative stable approximate solutions to ill-posed linear operator equations in a Hilbert space setting, we develop two novel non-negativity preserving iterative regularization methods. They are based on fixed point iterations in combination with preconditioning ideas. In contrast to the projected Landweber iteration, for which only weak convergence can be shown for the regularized solution when the noise level tends to zero, the introduced regularization methods exhibit strong convergence. There are presented convergence results, even for a combination of noisy right-hand side and imperfect forward operators, and for one of the approaches there are also convergence rates results. Specifically adapted discrepancy principles are used as a posteriori stopping rules of the established iterative regularization algorithms. For an application of the suggested new approaches, we consider a biosensor problem, which is modelled as a two dimensional linear Fredholm integral equation of the first kind. Several numerical examples, as well as a comparison with the projected Landweber method, are presented to show the accuracy and the acceleration effect of the novel methods. Case studies of a real data problem indicate that the developed methods can produce meaningful featured regularized solutions.

A joint bidiagonalization based iterative algorithm for large scale general-form Tikhonov regularization

Based on the joint bidiagonalization (JBD) process of the matrix pair {A,L}, an iterative regularization algorithm, called JBDQR, is proposed and developed for large scale linear discrete ill-posed problems in general-form Tikhonov regularization. It is proved that the JBDQR iterates take the form of attractive filtered generalized singular value decomposition (GSVD) expansions, where the filters are given explicitly and insightful. This result and a detailed analysis on it show that JBDQR must have the desired semi-convergence property, where the iteration number k plays the role of the regularization parameter. Embedded with the L-curve criterion or the discrepancy principle that is used to estimate the optimal k⁎ at which the semi-convergence occurs, JBDQR can compute a satisfying good regularized solution. JBDQR is theoretically solid and effective, and it is simple to implement. Numerical experiments confirm our results and the robustness of JBDQR.

Self-Supervised Light Field Reconstruction Using Shearlet Transform and Cycle Consistency

The image-based rendering approach using Shearlet Transform (ST) is one of the state-of-the-art Densely-Sampled Light Field (DSLF) reconstruction methods. It reconstructs Epipolar-Plane Images (EPIs) in image domain via an iterative regularization algorithm restoring their coefficients in shearlet domain. Consequently, the ST method tends to be slow because of the time spent on domain transformations for dozens of iterations. To overcome this limitation, this letter proposes a novel self-supervised DSLF reconstruction method, CycleST, which applies ST and cycle consistency to DSLF reconstruction. Specifically, CycleST is composed of an encoder-decoder network and a residual learning strategy that restore the shearlet coefficients of densely-sampled EPIs using EPI reconstruction and cycle consistency losses. Besides, CycleST is a self-supervised approach that can be trained solely on Sparsely-Sampled Light Fields (SSLFs) with small disparity ranges ($\leqslant$ 8 pixels). Experimental results of DSLF reconstruction on SSLFs with large disparity ranges (16 - 32 pixels) from two challenging real-world light field datasets demonstrate the effectiveness and efficiency of the proposed CycleST method. Furthermore, CycleST achieves ~ 9x speedup over ST, at least.

A weighted denoising method based on Bregman iterative regularization and gradient projection algorithms

A weighted Bregman-Gradient Projection denoising method, based on the Bregman iterative regularization (BIR) method and Chambolle’s Gradient Projection method (or dual denoising method) is established. Some applications to image denoising on a 1-dimensional curve, 2-dimensional gray image and 3-dimensional color image are presented.Compared with the main results of the literatures, the present numerical results of the proposed method are improved.

Self-prior strategy for organ reconstruction in fluorescence molecular tomography.

The purpose of this study is to propose a strategy for organ reconstruction in fluorescence molecular tomography (FMT) without prior information from other imaging modalities, and to overcome the high cost and ionizing radiation caused by the traditional structural prior strategy. The proposed strategy is designed as an iterative architecture to solve the inverse problem of FMT. In each iteration, a short time Fourier transform (STFT) based algorithm is used to extract the self-prior information in the space-frequency energy spectrum with the assumption that the regions with higher fluorescence concentration have larger energy intensity, then the cost function of the inverse problem is modified by the self-prior information, and lastly an iterative Laplacian regularization algorithm is conducted to solve the updated inverse problem and obtains the reconstruction results. Simulations and in vivo experiments on liver reconstruction are carried out to test the performance of the self-prior strategy on organ reconstruction. The organ reconstruction results obtained by the proposed self-prior strategy are closer to the ground truth than those obtained by the iterative Tikhonov regularization (ITKR) method (traditional non-prior strategy). Significant improvements are shown in the evaluation indexes of relative locational error (RLE), relative error (RE) and contrast-to-noise ratio (CNR). The self-prior strategy improves the organ reconstruction results compared with the non-prior strategy and also overcomes the shortcomings of the traditional structural prior strategy. Various applications such as metabolic imaging and pharmacokinetic study can be aided by this strategy.

Dual-channel fast iterative shrinkage-thresholding regularization algorithm for scanning radar forward-looking imaging

Regularization technology is an efficient and robust method to implement forward-looking imaging of scanning radar, and the selection of regularization parameter is quite important for the imaging performance. When the imaging scene contains sparse strong targets as well as a weak background, different regularization parameters should be adopted. A dual-channel fast iterative shrinkage-thresholding (D-FIST) regularization algorithm is proposed. First, by setting a threshold with mean segmentation method, the strong targets are extracted from the echo to formulate the data of channel I, and the entire data are included in channel II. The L-curve method is adopted to determine the regularization parameter for two channels data, respectively. Then, the data are processed by the FIST algorithm with corresponding parameters. At last, the imaging result is obtained by combining the results of two channels. Simulation experiments show that the imaging results of D-FIST are better than that of direct method, especially in a low SNR situation.

Light Field Reconstruction Using Shearlet Transform.

In this article we develop an image based rendering technique based on light field reconstruction from a limited set of perspective views acquired by cameras. Our approach utilizes sparse representation of epipolar-plane images (EPI) in shearlet transform domain. The shearlet transform has been specifically modified to handle the straight lines characteristic for EPI. The devised iterative regularization algorithm based on adaptive thresholding provides high-quality reconstruction results for relatively big disparities between neighboring views. The generated densely sampled light field of a given 3D scene is thus suitable for all applications which require light field reconstruction. The proposed algorithm compares favorably against state of the art depth image based rendering techniques and shows superior performance specifically in reconstructing scenes containing semi-transparent objects.

Reconstruction of Neural Activity from EEG Data Using Dynamic Spatiotemporal Constraints.

We present a novel iterative regularized algorithm (IRA) for neural activity reconstruction that explicitly includes spatiotemporal constraints, performing a trade-off between space and time resolutions. For improving the spatial accuracy provided by electroencephalography (EEG) signals, we explore a basis set that describes the smooth, localized areas of potentially active brain regions. In turn, we enhance the time resolution by adding the Markovian assumption for brain activity estimation at each time period. Moreover, to deal with applications that have either distributed or localized neural activity, the spatiotemporal constraints are expressed through [Formula: see text] and [Formula: see text] norms, respectively. For the purpose of validation, we estimate the neural reconstruction performance in time and space separately. Experimental testing is carried out on artificial data, simulating stationary and non-stationary EEG signals. Also, validation is accomplished on two real-world databases, one holding Evoked Potentials and another with EEG data of focal epilepsy. Moreover, responses of functional magnetic resonance imaging for the former EEG data have been measured in advance, allowing to contrast our findings. Obtained results show that the [Formula: see text]-based IRA produces a spatial resolution that is comparable to the one achieved by some widely used sparse-based estimators of brain activity. At the same time, the [Formula: see text]-based IRA outperforms other similar smooth solutions, providing a spatial resolution that is lower than the sparse [Formula: see text]-based solution. As a result, the proposed IRA is a promising method for improving the accuracy of brain activity reconstruction.

A direct method for photoacoustic tomography with inhomogeneous sound speed

The standard approach for photoacoustic imaging with variable speed of sound is time reversal, which consists of solving a well-posed final-boundary value problem for the wave equation backwards in time. This paper investigates the iterative Landweber regularization algorithm, where convergence is guaranteed by standard regularization theory, notably also in cases of trapping sound speed or for short measurement times. We formulate and solve the direct and inverse problem on the whole Euclidean space, which is common in standard photoacoustic imaging, but not for time reversal algorithms, where the problems are considered on a domain enclosed by the measurement devices. We formulate both the direct and adjoint photoacoustic operator as the solution of an interior and an exterior differential equation which are coupled by transmission conditions. The former is solved numerically using a Galerkin scheme in space and finite difference discretization in time, while the latter consists of solving a boundary integral equation. We therefore use a boundary element method/finite element method approach for numerical solution of the forward operators. We analyze this method, prove convergence, and provide numerical tests. Moreover, we compare the approach to time reversal.