Iterative Shrinkage-thresholding Algorithm Research Articles

Photo-acoustic tomographic image reconstruction from reduced data using physically inspired regularization

We propose a model-based image reconstruction method for photoacoustic tomography (PAT) involving a novel form of regularization and demonstrate its ability to recover good quality images from significantly reduced size datasets. The regularization is constructed to suit the physical structure of typical PAT images. We construct it by combining second-order derivatives and intensity into a non-convex form to exploit a structural property of PAT images that we observe: in PAT images, high intensities and high second-order derivatives are jointly sparse. The specific form of regularization constructed here is a modification of the form proposed for fluorescence image restoration. This regularization is combined with a data fidelity cost, and the required image is obtained as the minimizer of this cost. As this regularization is non-convex, the efficiency of the minimization method is crucial in obtaining artifact-free reconstructions. We develop a custom minimization method for efficiently handling this non-convex minimization problem. Further, as non-convex minimization requires a large number of iterations and the PAT forward model in the data-fidelity term has to be applied in the iterations, we propose a computational structure for efficient implementation of the forward model with reduced memory requirements. We evaluate the proposed method on both simulated and real measured data sets and compare them with a recent reconstruction method that is based on a well-known fast iterative shrinkage threshold algorithm (FISTA).

Smoothing accelerated algorithm for constrained nonsmooth convex optimization problems

In this paper, we consider a class of constrained nonsmooth convex optimization problems, which have wide applications in signal processing, imaging recovery, machine learning, etc. Recently, theoretical analysis and numerical experiments show that the extrapolation can improve the convergence rate of the algorithms efficiently, which let the proximal gradient algorithm with extrapolation have significant advantages in solving large scale optimization problems. Motivated by the smoothing techniques, and the fast iterative shrinkage-thresholding algorithm (FISTA) proposed by Beck and Teboulle, we bring forward a new accelerated algorithm for solving the considered constrained nonsmooth convex optimization problems. We prove that any accumulated point of the algorithm is an optimal solution of the problem. In the analysis of the algorithm, we consider several different update methods for the smoothing parameter, and give the global convergence rate $O(\\ln~k/k)$ on the objective function values. Moreover, we analyze the convergence rate of the difference on the iterates $\\lim_{k\\rightarrow}\\|x^{k+1}-x^k\\|=0$. Finally, our numerical experiments illustrate the good performance of the proposed algorithm in solving two classes of sparse optimization problems and the positive influence of the extrapolation on the convergence rate.

Ridge-Aware Weighted Sparse Time-Frequency Representation

The ideal time-frequency (TF) representation which distributes the total energy along the instantaneous frequency (IF) of a signal is essentially sparse. Motivated by the weighted sparse representation of the signal, we propose the ridge-aware weighted sparse TF representation (RWSTF) which involves some properties an ideal TF representation should satisfy, such as, highly concentrated TF representation, the signal reconstruction and acceptable computational cost. Based on a basic sparse TF model, we firstly use a weighted strategy to effectively highlight the TF ridges even for the weak components, then fast iterative shrinkage thresholding algorithm (FISTA) is applied to obtain an efficient numerical approximation for solving the model. Furthermore, we introduce a k-sparsity strategy for the adaptive selection of the regularization parameter. A simulation study shows that the proposed method not only has higher energy concentration, but also performs better on signal denoising than other standard TF approaches, especially for the signals with fast varying IF. Two real life examples confirm the potential of the proposed method.

Compressive sensing MR imaging based on adaptive tight frame and reference image

Compressive sensing magnetic resonance (MR) imaging is aimed at achieving high-quality MR image reconstruction by undersampling K-space data. It is crucial to explore prior information since compressive sensing MR imaging relies heavily on some prior assumptions, such as signal's sparse property. In this study, in order to explore the prior information fully, an improved MR image reconstruction model based on compressive sensing theory is proposed, named reference image MR imaging with adaptive tight frame. In the proposed model, an adaptive tight frame is involved to explore the sparse prior information adapt to MR images and the similarity prior information to the target image. Meanwhile, improved adaptive weighting parameters are used to trade off the sparsity between the regions with much similarity and that of little similarity. In addition, the smoothing-based fast iterative shrinkage-threshold algorithm is utilised to tackle the optimisation problem so as to speed up imaging. The experimental results demonstrate that the proposed MR image reconstruction method outperforms some state-of-the-art methods in terms of quantitative results.

Early-photon reflectance fluorescence molecular tomography for small animal imaging: Mathematical model and numerical experiment.

The paper presents an original approach to time-domain reflectance fluorescence molecular tomography (FMT) of small animals. It is based on the use of early arriving photons and state-of-the-art compressed-sensing-like reconstruction algorithms and aims to improve the spatial resolution of fluorescent images. We deduce the fundamental equation that models the imaging operator and derive analytical representations for the sensitivity functions which are responsible for the reconstruction of the fluorophore absorption coefficient. The idea of fluorescence lifetime tomography with our approach is also discussed. We conduct a numerical experiment on 3D reconstruction of box phantoms with spherical fluorescent inclusions of small diameters. For modeling measurement data and constructing the sensitivity matrix we assume a virtual fluorescence tomograph with a scanning fiber probe that illuminates and collects light in reflectance geometry. It provides for large source-receiver separations which correspond to the macroscopic regime. Two compressed-sensing-like reconstruction algorithms are used to solve the inverse problem. These are the algebraic reconstruction technique with total variation regularization and our modification of the fast iterative shrinkage-thresholding algorithm. Results of our numerical experiment show that our approach is capable of achieving as good spatial resolution as 0.2 mm and even better at depths to 9 mm inclusive.

Brain functional network reconstruction based on compressed sensing and fast iterative shrinkage-thresholding algorithm

The construction of brain functional network based on resting-state functional magnetic resonance imaging (fMRI) is an effective method to reveal the mechanism of human brain operation, but the common brain functional network generally contains a lot of noise, which leads to wrong analysis results. In this paper, the least absolute shrinkage and selection operator (LASSO) model in compressed sensing is used to reconstruct the brain functional network. This model uses the sparsity of L1-norm penalty term to avoid over fitting problem. Then, it is solved by the fast iterative shrinkage-thresholding algorithm (FISTA), which updates the variables through a shrinkage threshold operation in each iteration to converge to the global optimal solution. The experimental results show that compared with other methods, this method can improve the accuracy of noise reduction and reconstruction of brain functional network to more than 98%, effectively suppress the noise, and help to better explore the function of human brain in noisy environment.

Comparison of four deblurring methods for streak image of streak tube imaging lidar

In this study, the Richardson–Lucy (R–L) iteration, Tikhonov (T–K) regularization, and fast iterative shrinkage-thresholding algorithm (FISTA) are first used to deblur streak images. The deblur performances of these three methods are compared with that of Wiener deconvolution. The spatial resolutions of Wiener deconvolution, R–L iteration, T–K regularization, and FISTA improved from 9 mm to 4.5, 5, 5, and 5 mm, respectively. The root-mean-square errors (RMSE) of one -, two -, and three-plane targets decreased effectively using these three deblur methods. The R–L iteration performs the best in terms of the RMSE for the two-plane target. The effects of the parameters used in each method are investigated.

Negentropy-Based Sparsity-Promoting Reconstruction with Fast Iterative Solution from Noisy Measurements.

Compressed sensing provides an elegant framework for recovering sparse signals from compressed measurements. This paper addresses the problem of sparse signal reconstruction from compressed measurements that is more robust to complex, especially non-Gaussian noise, which arises in many applications. For this purpose, we present a method that exploits the maximum negentropy theory to promote the adaptability to noise. This problem is formalized as a constrained minimization problem, where the objective function is the negentropy of measurement error with sparse constraint -norm. On the minimization issue of the problem, although several promising algorithms have been proposed in the literature, they are very computationally demanding and thus cannot be used in many practical situations. To improve on this, we propose an efficient algorithm based on a fast iterative shrinkage-thresholding algorithm that can converge fast. Both the theoretical analysis and numerical experiments show the better accuracy and convergent rate of the proposed method.

Full waveform inversion by proximal Newton method using adaptive regularization

SUMMARY Regularization is necessary for solving non-linear ill-posed inverse problems arising in different fields of geosciences. The base of a suitable regularization is the prior expressed by the regularizer, which can be non-adaptive or adaptive (data-driven), smooth or non-smooth, variational-based or not. Nevertheless, tailoring a suitable and easy-to-implement prior for describing geophysical models is a non-trivial task. In this paper, we propose two generic optimization algorithms to implement arbitrary regularization in non-linear inverse problems such as full-waveform inversion (FWI), where the regularization task is recast as a denoising problem. We assess these optimization algorithms with the plug-and-play block matching (BM3D) regularization algorithm, which determines empirical priors adaptively without any optimization formulation. The non-linear inverse problem is solved with a proximal Newton method, which generalizes the traditional Newton step in such a way to involve the gradients/subgradients of a (possibly non-differentiable) regularization function through operator splitting and proximal mappings. Furthermore, it requires to account for the Hessian matrix in the regularized least-squares optimization problem. We propose two different splitting algorithms for this task. In the first, we compute the Newton search direction with an iterative method based upon the first-order generalized iterative shrinkage-thresholding algorithm (ISTA), and hence Newton-ISTA (NISTA). The iterations require only Hessian-vector products to compute the gradient step of the quadratic approximation of the non-linear objective function. The second relies on the alternating direction method of multipliers (ADMM), and hence Newton-ADMM (NADMM), where the least-squares optimization subproblem and the regularization subproblem in the composite objective function are decoupled through auxiliary variable and solved in an alternating mode. The least-squares subproblem can be solved with exact, inexact, or quasi-Newton methods. We compare NISTA and NADMM numerically by solving FWI with BM3D regularization. The tests show promising results obtained by both algorithms. However, NADMM shows a faster convergence rate than NISTA when using L-BFGS to solve the Newton system.

Subpixel image registration regularised by and norms

In this study, the authors propose a subpixel image registration framework that detects and matches feature points. Rigid and nonrigid registration models are employed to solve the problem of subpixel image registration problem. A rigid registration model based on the l 2 norm is proposed to regularise the rotation coefficients using the indicator function to estimate the rigid transformation parameters. The latter estimation simplified is made easy by the reduction in the rigid transformation from two dimensions to one dimension. Furthermore, a non-rigid registration model based on the l 1 and l 2 norms is proposed to estimate the elastic coefficients of the compact support radial basis functions. Due to the linear representation of the transformation function, the rigid and nonrigid subpixel image registration models can be solved efficiently using the fast iterative shrinkage-thresholding algorithm. Experiments on a demosaicing data set, the ocean of remote sensing data set, a brain data set and the fundus image registration data set show that the proposed rigid and non-rigid registration models can accurately perform subpixel image registration.

Dynamic Multivariate Functional Data Modeling via Sparse Subspace Learning

ABSTRACT Multivariate functional data from a complex system are naturally high-dimensional and have a complex cross-correlation structure. The complexity of data structure can be observed as that (1) some functions are strongly correlated with similar features, while some others may have almost no cross-correlations with quite diverse features; and (2) the cross-correlation structure may also change over time due to the system evolution. With this regard, this article presents a dynamic subspace learning method for multivariate functional data modeling. In particular, we consider that different functions come from different subspaces, and only functions of the same subspace have cross-correlations with each other. The subspaces can be automatically formulated and learned by reformatting the problem as a sparse regression. By allowing but regularizing the regression change over time, we can describe the cross-correlation dynamics. The model can be efficiently estimated by the fast iterative shrinkage-thresholding algorithm, and the features of each subspace can be extracted using the smooth multi-channel functional principal component analysis. Some theoretical properties of the model are presented. Numerical studies, together with case studies, demonstrate the efficiency and applicability of the proposed methodology.

Anti-sparse representation for structural model updating using l ∞ norm regularization

Finite element (FE) model based structural damage detection (SDD) methods play vital roles in effectively locating and quantifying structural damages. Among these methods, structural model updating should be conducted before SDD to obtain benchmark models of real structures. However, the characteristics of updating parameters are not reasonably considered in existing studies. Inspired by the lll∞ norm regularization, a novel anti-sparse representation method is proposed for structural model updating in this study. Based on sensitivity analysis, both frequencies and mode shapes are used to define an objective function at first. Then, by adding l∞ norm penalty, an optimization problem is established for structural model updating. As a result, the optimization problem can be solved by the fast iterative shrinkage thresholding algorithm (FISTA). Moreover, comparative studies with classical regularization strategy, i.e. the l2 norm regularization method, are conducted as well. To intuitively illustrate the effectiveness of the proposed method, a 2-DOF spring-mass model is taken as an example in numerical simulations. The updating results show that the proposed method has a good robustness to measurement noises. Finally, to further verify the applicability of the proposed method, a six-storey aluminum alloy frame is designed and fabricated in laboratory. The added mass on each storey is taken as updating parameter. The updating results provide a good agreement with the true values, which indicates that the proposed method can effectively update the model parameters with a high accuracy.

Pruning multi-view stereo net for efficient 3D reconstruction

How can we perform an efficient 3D reconstruction with high accuracy and completeness, in the presence of non-Lambertian surface and low textured regions? This paper aims at fast quality 3D reconstruction, best near real time. While deep learning approaches perform very well in multi-view stereo (MVS), the high complexity of models makes them inapplicable in practical applications. Few works were explored to accelerate deep learning-based 3D reconstruction approaches. In this paper, we take an unprecedented attempt to compress and accelerate these models via pruning their redundant parameters. We introduce an efficient channel pruning method for 2D convolutional neural networks (CNNs) based on a mixed back propagation process, where a soft mask is learned to prune the channels using a fast iterative shrinkage-thresholding algorithm. While in 3D CNNs, we train a large multi-scale CNNs architecture and observe that only utilizing one module enough for the 3D reconstruction, which can still maintain the performance of the full-precision model. We achieve an efficient MVS reconstruction system up to 2 times faster, in contrast to the state-of-the-arts, while maintaining comparable model accuracy and even better completeness.

Fraction-variant beam orientation optimization for intensity-modulated proton therapy.

To achieve a superior balance between dosimetry and the delivery efficiency of intensity-modulated proton therapy (IMPT) using as few beams as possible in a single fraction, we optimally vary beams in different fractions. In the optimization, 400~800 feasible noncoplanar beams were included in the candidate pool. For each beam, the doses of all scanning spots covering the target volume and a margin were calculated. The fraction-variant beam orientation optimization (FVBOO) problem was formulated to include three terms: two quadratic dose fidelity terms to penalize the deviation of planning target volume fractional dose and organs at risk (OAR) cumulative doses from prescription, respectively; an L2,1/2-norm group sparsity term to control the number of active beams per fraction to between 1 and 4. The Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) was applied to solve this problem. FVBOO was tested on a patient with base-of-skull (BOS) tumor of 5 fractions (5f) and 30 fractions (30f) with an average number of active beams per fraction varying between 4 and 1. In addition, one bilateral head-and-neck (H&N) patient, and one esophageal cancer (ESG) patient of 30f were tested with about three active beams per fraction. The results were compared with IMPT plans that use fixed beams in each fraction. The fixed beams were selected using the group sparsity term with a fraction-invariant BOO (FIBOO) constraint. Varying beams were chosen in either the 5f or 30f FVBOO plans. While similar number of beams per fraction was selected as the FIBOO plan, the FVBOO plans were able to spare the OARs better, with an average reduction of [Dmean, Dmax] from the FIBOO plans by [0.85, 2.08] Relative Biological Effective Gy (GyRBE) in the 5f plan and [1.87, 4.06]GyRBE in the 30f plans. While reducing the number of beams per fraction in the BOS patient, a three-beam/fraction 5f FVBOO plan performs comparably as the four-beam FIBOO plan and a two-beam/fraction 30f FVBOO plan still provides superior dosimetry. Fraction-variant beam orientation optimization allows the utilization of a larger beam solution space for superior dose distribution in IMPT while maintaining a practical number of beams in each fraction.

A Combined Method for Time-Varying Parameter Identification Based on Variational Mode Decomposition and Generalized Morse Wavelet

Time-varying parameter identification is essential for structural health monitoring and performance evaluation. In this paper, a combined method based on the variational mode decomposition and generalized Morse wavelet is proposed to identify the structural time-varying parameters. Based on the sparse property of structural response signals in wavelet domain, a fast iterative shrinkage-thresholding algorithm is adopted to reduce the noise. Then the de-noised signal is decomposed into multi- modes by the variational mode decomposition, and the generalized Morse wavelet is performed to identify the instantaneous frequency. To validate the proposed method, a numerical example including different frequency variations is studied. Experimental validations of a moving vehicle across a bridge and a time-varying cable system considering two patterns of cable tension variations in the laboratory are carried out to investigate the capability of the proposed approach. It is confirmed that the proposed approach can effectively perform the signal decomposition, while identifying the instantaneous frequencies of the time-varying systems accurately.

A Sparse Spike Deconvolution Algorithm Based on a Recurrent Neural Network and the Iterative Shrinkage-Thresholding Algorithm

Conventional sparse spike deconvolution algorithms that are based on the iterative shrinkage-thresholding algorithm (ISTA) are widely used. The aim of this type of algorithm is to obtain accurate seismic wavelets. When this is not fulfilled, the processing stops being optimum. Using a recurrent neural network (RNN) as deep learning method and applying backpropagation to ISTA, we have developed an RNN-like ISTA as an alternative sparse spike deconvolution algorithm. The algorithm is tested with both synthetic and real seismic data. The algorithm first builds a training dataset from existing well-logs seismic data and then extracts wavelets from those seismic data for further processing. Based on the extracted wavelets, the new method uses ISTA to calculate the reflection coefficients. Next, inspired by the backpropagation through time (BPTT) algorithm, backward error correction is performed on the wavelets while using the errors between the calculated reflection coefficients and the reflection coefficients corresponding to the training dataset. Finally, after performing backward correction over multiple iterations, a set of acceptable seismic wavelets is obtained, which is then used to deduce the sequence of reflection coefficients of the real data. The new algorithm improves the accuracy of the deconvolution results by reducing the effect of wrong seismic wavelets that are given by conventional ISTA. In this study, we account for the mechanism and the derivation of the proposed algorithm, and verify its effectiveness through experimentation using theoretical and real data.

Adaptive estimation and sparse sampling for graph signals in alpha-stable noise

In the graph signal processing literature, most methods were developed based on the assumption of Gaussian noise since it can lead to computationally efficient and mathematically tractable solutions. Unfortunately, the Gaussian distribution cannot capture the sharp spikes and tail heaviness of the signal noise in various natural phenomena. The α-stable distribution is a generalization of the Gaussian distribution and is a more appropriate model for such impulsive heavy-tailed noise. In this paper, we consider the problem of adaptive estimation and sparse sampling for signals defined over graphs in the presence of impulsive α-stable noise. To tackle the problems associated with α-stable noise, the graph signal estimation problem is formulated as a minimum dispersion (MD)-based optimization. A novel adaptive least mean pth power (LMP) algorithm is proposed for robust estimation of band-limited graph signals from partial observations in α-stable noise environments. The mean square performance of the proposed LMP algorithm is theoretically analyzed. To handle the case of band-limited graph signals with unknown and time-varying bandwidth and spectral contents, the sparse nature of the Graph Fourier transform of the signal is exploited to develop an adaptive graph sampling technique. Specifically, the proposed sparse sampling technique identifies the spectral signal support via sparse online estimation building on the iterative shrinkage-thresholding algorithm. Numerical simulation studies are presented to corroborate the performance advantages of the proposed adaptive estimation and sparse sampling algorithms.

Spectral Mixture Model Inspired Network Architectures for Hyperspectral Unmixing

In many statistical hyperspectral unmixing approaches, the unmixing task is essentially an optimization problem given a defined linear or nonlinear spectral mixture model. However, most of the model inference algorithms require a time-consuming iterative procedure. On the other hand, neural networks have been recently used to estimate abundances given some training samples, or directly estimate endmembers and abundances simultaneously in an unsupervised setting. However, their disadvantages are clear: lack of interpretability and reliance on the large training set. Model-inspired neural networks are constructed by the problem model and its corresponding inference algorithm. It incorporates the prior knowledge of physical model and algorithm into network architecture, combining the advantages of model-based and learning-based methods. This article deeply unfolds the linear mixture model and the corresponding iterative shrinkage-thresholding algorithm (ISTA) to build two unmixing network architectures. The first assumes that the set of endmembers are known, and the deep unfolded ISTA model is only for abundance estimation; and the second is used for blind unmixing to estimate both endmembers and abundances at the same time. The networks can be trained by supervised and unsupervised schemes, respectively, with a small-size training set, and then, unmixing becomes a feedforward process, which is very fast since no iteration is required. The experimental results show their competitive performance compared with the state-of-the-art unmixing approaches.

Sparsifying spherical radial basis functions based regional gravity models

ABSTRACT Regularization is often necessary in parameter estimation of spherical radial basis function (SRBF) based regional gravity modelling. In this work, a new regularization method, namely the L1-norm regularization, also called the least absolute shrinkage selection operator (Lasso), is employed. This is different from the commonly used L2-norm regularization, also called the Tikhonov regularization. The solution produced by Lasso is sparse, namely with part of the parameters being exactly zero. A sparse model would be beneficial in terms of better interpretability and improved variable assigning efficiency. The optimization problem involving the L1-norm regularization is solved by employing a highly efficient numerical algorithm called fast iterative shrinkage threshold algorithm (FISTA). The hyperparameter, namely the regularization coefficient, is selected through 10-fold cross validation. Simple closed-loop simulation is conducted using EGM2008. Without loss of generality, the point-mass method, a special kind of SRBF kernels, is employed. The accuracy and the sparsity of the Lasso-based SRBF model is demonstrated in this simulation. In this work, the Tikhonov regularization method outperforms the L1-norm regularizations method about 20% in regions with rough gravity field features, in regions with smooth gravity field features two regularization methods perform almost equivalently in terms of modelling accuracy. Sparsity of Lasso is obvious, to be more specific, sparsity ratio, namely the ratios of the number of zero parameters to that of the original full parameters, are greater than 0.5.

Smoothing for continuous dynamical state space models with sampled system coefficients based on sparse kernel learning

A new smoother for a continuous dynamical state space model with sampled system coefficients is proposed. This is completely different from conventional approaches, such as Rauch–Tung–Striebel smoother. In the proposed method, the state vector as a continuous function of time is represented by kernel models. The state process model, namely the differential equation, is treated as part of the measurement model at the sampling instants of the system coefficients. Sparse solution of the kernel weights is obtained through a special regularization strategy called the Lasso estimator. The optimization problem appearing in the Lasso estimation is solved by the fast iterative shrinkage threshold algorithm. The hyperparameters involved, namely the kernel widths and the regularization coefficients, are selected objectively through generalized cross-validation or corrected Akaike information criterion tailored to the Lasso estimator. A simple two-dimension example is employed in the simulation to demonstrate the application and also the performance of the proposed method. It is shown that the proposed method could provide state vector estimates with satisfactory accuracy not only at the sampling instants of the observations but also at any other instants. The sparsity of the solution could also be clearly seen in the experiment. The proposed method can be viewed as an alternative smoothing method, rather than a replacement for conventional smoothers, due to the difficult model tuning and increased computation load.