Iterative Soft Thresholding Algorithm Research Articles

Deep Learning-Based Approximated Observation Sparse SAR Imaging via Complex-Valued Convolutional Neural Network

Sparse synthetic aperture radar (SAR) imaging has demonstrated excellent potential in image quality improvement and data compression. However, conventional observation matrix-based methods suffer from high computational overhead, which is hard to use for real data processing. The approximated observation sparse SAR imaging method relieves the computation pressure, but it needs to manually set the parameters to solve the optimization problem. Thus, several deep learning (DL) SAR imaging methods have been used for scene recovery, but many of them employ dual-path networks. To better leverage the complex-valued characteristics of echo data, in this paper, we present a novel complex-valued convolutional neural network (CNN)-based approximated observation sparse SAR imaging method, which is a single-path DL network. Firstly, we present the approximated observation-based model via the chirp-scaling algorithm (CSA). Next, we map the process of the iterative soft thresholding (IST) algorithm into the deep network form, and design the symmetric complex-valued CNN block to achieve the sparse recovery of large-scale scenes. In comparison to matched filtering (MF), the approximated observation sparse imaging method, and the existing DL SAR imaging methods, our complex-valued network model shows excellent performance in image quality improvement especially when the used data are down-sampled.

A fused LASSO operator for fast 3D magnetic particle imaging reconstruction

Objective. Magnetic particle imaging (MPI) is a promising imaging modality that leverages the nonlinear magnetization behavior of superparamagnetic iron oxide nanoparticles to determine their concentration distribution. Previous optimization models with multiple regularization terms have been proposed to achieve high-quality MPI reconstruction, but these models often result in increased computational burden, particularly for dense gridding 3D fields of view. In order to achieve faster reconstruction speeds without compromising reconstruction quality, we have developed a novel fused LASSO operator, total sum-difference (TSD), which effectively captures the sparse and smooth priors of MPI images. Methods. Through an analysis-synthesis equivalence strategy and a constraint smoothing strategy, the TSD regularized model was solved using the fast iterative soft-thresholding algorithm (FISTA). The resulting reconstruction method, TSD-FISTA, boasts low computational complexity and quadratic convergence rate over iterations. Results. Experimental results demonstrated that TSD-FISTA required only 10% and 37% of the time to achieve comparable or superior reconstruction quality compared to commonly used fused LASSO-based alternating direction method of multipliers and Tikhonov-based algebraic reconstruction techniques, respectively. Significance. TSD-FISTA shows promise for enabling real-time 3D MPI reconstruction at high frame rates for large fields of view.

[formula omitted] sparsity regularization for nonlinear ill-posed problems

In this paper, the α‖⋅‖ℓ1−β‖⋅‖ℓ2 sparsity regularization with parameters α≥β≥0 is studied for nonlinear ill-posed inverse problems. The well-posedness of the regularization is investigated. Compared to the case where α>β≥0, the results for the case α=β>0 are weaker due to the lack of coercivity and Radon-Riesz property of the regularization term. Under certain conditions on the nonlinearity of F, sparsity is shown for every minimizer of the α‖⋅‖ℓ1−β‖⋅‖ℓ2 regularized inverse problem. Moreover, for the case α>β≥0, convergence rates O(δ12) and O(δ) are proved for the regularized solution toward a sparse exact solution, under different yet commonly adopted conditions on the nonlinearity of F. The iterative soft thresholding algorithm is shown to be useful to solve the α‖⋅‖ℓ1−β‖⋅‖ℓ2 regularized problem for nonlinear ill-posed equations. Numerical results illustrate the efficiency of the proposed method.

PISTA: Preconditioned Iterative Soft Thresholding Algorithm for Graphical Lasso

pISTA: Preconditioned Iterative Soft Thresholding Algorithm for Graphical Lasso

Lasso-based state estimation for cyber-physical systems under sensor attacks

The development of algorithms for secure state estimation in vulnerable cyber-physical systems has been gaining attention in the last years. A consolidated assumption is that an adversary can tamper a relatively small number of sensors. In the literature, block-sparsity methods exploit this prior information to recover the attack locations and the state of the system. In this paper, we propose an alternative, Lasso-based approach and we analyse its effectiveness. In particular, we theoretically derive conditions that guarantee successful attack/state recovery, independently of established time sparsity patterns. Furthermore, we develop a sparse state observer, by starting from the iterative soft thresholding algorithm for Lasso, to perform online estimation. Through several numerical experiments, we compare the proposed methods to the state-of-the-art algorithms.

Compressed Sensing Techniques Applied to Medical Images Obtained with Magnetic Resonance

The fast and reliable processing of medical images is of paramount importance to adequately generate data to feed machine learning algorithms that can prevent and diagnose health issues. Here, different compressed sensing techniques applied to magnetic resonance imaging are benchmarked as a means to reduce the acquisition time spent in the collection of data and signals that form the image. It is shown that by using these techniques, it is possible to reduce the number of signals needed and, therefore, substantially decrease the time to acquire the measurements. To this end, different algorithms are considered and compared: the iterative re-weighted least squares, the iterative soft thresholding algorithm, the iterative hard thresholding algorithm, the primal dual algorithm and the log barrier algorithm. Such algorithms have been implemented in different analysis programs that have been used to perform the reconstruction of the images, and it was found that the iterative soft thresholding algorithm gives the optimal results. It is found that the images obtained with this algorithm have lower quality than the original ones, but in any case, the quality should be good enough to distinguish each body structure and detect any health problems under an expert evaluation and/or statistical analysis.

A projected gradient method for nonlinear inverse problems with 𝛼ℓ1 − 𝛽ℓ2 sparsity regularization

Abstract The non-convex α ⁢ ∥ ⋅ ∥ ℓ 1 − β ⁢ ∥ ⋅ ∥ ℓ 2 \alpha\lVert\,{\cdot}\,\rVert_{\ell_{1}}-\beta\lVert\,{\cdot}\,\rVert_{\ell_{2}} ( α ≥ β ≥ 0 \alpha\geq\beta\geq 0 ) regularization is a new approach for sparse recovery. A minimizer of the α ⁢ ∥ ⋅ ∥ ℓ 1 − β ⁢ ∥ ⋅ ∥ ℓ 2 \alpha\lVert\,{\cdot}\,\rVert_{\ell_{1}}-\beta\lVert\,{\cdot}\,\rVert_{\ell_{2}} regularized function can be computed by applying the ST-( α ⁢ ℓ 1 − β ⁢ ℓ 2 \alpha\ell_{1}-\beta\ell_{2} ) algorithm which is similar to the classical iterative soft thresholding algorithm (ISTA). Unfortunately, It is known that ISTA converges quite slowly, and a faster alternative to ISTA is the projected gradient (PG) method. Nevertheless, the current applicability of the PG method is limited to linear inverse problems. In this paper, we extend the PG method based on a surrogate function approach to nonlinear inverse problems with the α ⁢ ∥ ⋅ ∥ ℓ 1 − β ⁢ ∥ ⋅ ∥ ℓ 2 \alpha\lVert\,{\cdot}\,\rVert_{\ell_{1}}-\beta\lVert\,{\cdot}\,\rVert_{\ell_{2}} ( α ≥ β ≥ 0 \alpha\geq\beta\geq 0 ) regularization in the finite-dimensional space R n \mathbb{R}^{n} . It is shown that the presented algorithm converges subsequentially to a stationary point of a constrained Tikhonov-type functional for sparsity regularization. Numerical experiments are given in the context of a nonlinear compressive sensing problem to illustrate the efficiency of the proposed approach.

Generalization error bounds for iterative recovery algorithms unfolded as neural networks

Abstract Motivated by the learned iterative soft thresholding algorithm (LISTA), we introduce a general class of neural networks suitable for sparse reconstruction from few linear measurements. By allowing a wide range of degrees of weight-sharing between the flayers, we enable a unified analysis for very different neural network types, ranging from recurrent ones to networks more similar to standard feedforward neural networks. Based on training samples, via empirical risk minimization, we aim at learning the optimal network parameters and thereby the optimal network that reconstructs signals from their low-dimensional linear measurements. We derive generalization bounds by analyzing the Rademacher complexity of hypothesis classes consisting of such deep networks, that also take into account the thresholding parameters. We obtain estimates of the sample complexity that essentially depend only linearly on the number of parameters and on the depth. We apply our main result to obtain specific generalization bounds for several practical examples, including different algorithms for (implicit) dictionary learning, and convolutional neural networks.

A convergence analysis for projected fast iterative soft-thresholding algorithm under radial sampling MRI

Radial sampling is a fast magnetic resonance imaging technique. Further imaging acceleration can be achieved with undersampling but how to reconstruct a clear image with fast algorithm is still challenging. Previous work has shown the advantage of removing undersampling image artifacts using the tight-frame sparse reconstruction model. This model was further solved with a projected fast iterative soft-thresholding algorithm (pFISTA). However, the convergence of this algorithm under radial sampling has not been clearly set up. In this work, the authors derived a theoretical convergence condition for this algorithm. This condition was approximated by estimating the maximal eigenvalue of reconstruction operators through the power iteration. Based on the condition, an optimal step size was further suggested to allow the fastest convergence. Verifications were made on the prospective in vivo data of static brain imaging and dynamic contrast-enhanced liver imaging, demonstrating that the recommended parameter allowed fast convergence in radial MRI.

A fast data-driven method for inverse microphone array signal processing

Microphone arrays have long been used to characterize and locate sound sources. However, existing algorithms for processing the signals are computationally expensive and, consequently, different methods need to be explored. Recently, the trained iterative soft thresholding algorithm (TISTA), a data-driven solver for inverse problems, was shown to improve on existing approaches. Here, a more in-depth analysis of its robustness and frequency dependence is provided using synthesized as well as real measurement data. It is demonstrated that TISTA yields favorable results in comparison to a covariance matrix fitting inverse method, especially for large numbers of sources.

Undersampled MRI reconstruction based on spectral graph wavelet transform

Compressed sensing magnetic resonance imaging (CS-MRI) has exhibited great potential to accelerate magnetic resonance imaging if an image can be sparsely represented. How to sparsify the image significantly affects the reconstruction quality of images. In this paper, a spectral graph wavelet transform (SGWT) is introduced to sparsely represent magnetic resonance images in iterative image reconstructions. The SGWT is achieved by extending the traditional wavelets transform to the signal defined on the vertices of the weighted graph, i.e. the spectral graph domain. This SGWT uses only the connectivity information encoded in the edge weights, and does not rely on any other attributes of the vertices. Therefore, SGWT can be defined and calculated for any domain where the underlying relations between data locations can be represented by a weighted graph. Furthermore, we present a Chebyshev polynomial approximation algorithm for fast computing this SGWT transform. The l1 norm regularized CS-MRI reconstruction model is introduced and solved by the projected iterative soft-thresholding algorithm to verify its feasibility. Numerical experiment results demonstrate that our proposed method outperforms several state-of-the-art sparsify transforms in terms of suppressing artifacts and achieving lower reconstruction errors on the tested datasets.

Compressive sensing reconstruction for rolling bearing vibration signal based on improved iterative soft thresholding algorithm

In order to solve the problem of high data transmission pressure and storage cost caused by collecting massive data in rolling bearing health monitoring, a new method of acquisition and reconstruction based on compressive sensing is introduced in this paper, which can recover the original signal accurately with a small amount of data. However, the traditional iterative soft thresholding algorithm (ISTA) has some shortcomings in the reconstruction process of vibration signal, such as fixed gradient step size, slow convergence speed, and poor reconstruction accuracy. To overcome this problem, an improved ISTA (IISTA) is put forward. Firstly, an acceleration operator considering step size is proposed to estimate the gradient, which can accelerate convergence speed. Then for breaking the limitation of fixed internal gradient step size, a bidirectional search principle is introduced to backtrack its iteration step size. Finally, the constraint of quadratic approximation model on the reconstruction objective function is used to adaptively determine optimal iteration step size. The effectiveness of the approach is verified by the reconstruction of bearing vibration signals in different health states. The results show that the proposed method outperforms the conventional ISTA in terms of reconstruction accuracy and efficiency.

Contrastive Domain Adaptation-Based Sparse SAR Target Classification under Few-Shot Cases

Due to the imaging mechanism of synthetic aperture radar (SAR), it is difficult and costly to acquire abundant labeled SAR images. Moreover, a typical matched filtering (MF) based image faces the problems of serious noise, sidelobes, and clutters, which will bring down the accuracy of SAR target classification. Different from the MF-based result, a sparse image shows better quality with less noise and higher image signal-to-noise ratio (SNR). Therefore, theoretically using it for target classification will achieve better performance. In this paper, a novel contrastive domain adaptation (CDA) based sparse SAR target classification method is proposed to solve the problem of insufficient samples. In the proposed method, we firstly construct a sparse SAR image dataset by using the complex image based iterative soft thresholding (BiIST) algorithm. Then, the simulated and real SAR datasets are simultaneously sent into an unsupervised domain adaptation framework to reduce the distribution difference and obtain the reconstructed simulated SAR images for subsequent target classification. Finally, the reconstructed simulated images are manually labeled and fed into a shallow convolutional neural network (CNN) for target classification along with a small number of real sparse SAR images. Since the current definition of the number of small samples is still vague and inconsistent, this paper defines few-shot as less than 20 per class. Experimental results based on MSTAR under standard operating conditions (SOC) and extended operating conditions (EOC) show that the reconstructed simulated SAR dataset makes up for the insufficient information from limited real data. Compared with other typical deep learning methods based on limited samples, our method is able to achieve higher accuracy especially under the conditions of few shots.

A Hybrid Iterative Thresholding Method for Solving Sparsity-Regularized Linear Inverse Problems and Its Application in Seismic Sparse-Spike Deconvolution

Sparsity-regularized linear inverse problem has been served as the base in many disciplines, such as remote sensing imaging, image processing and analysis, seismic deconvolution, compressed sensing, medical imaging, and so forth. Iterative hard thresholding algorithm (IHTA) and iterative soft thresholding algorithm (ISTA) are two frequently used methods to solve sparsity-regularized linear inverse problems. They are also the basic unit of other more complex methods. IHTA and ISTA are derived under steepest descent method. I.e., iteratively perform gradient descent and thresholding shrinkage. Steepest descent method is a first order algorithm, which is a powerful way to solve optimization due to its relatively simple implementation. However, a known issue of first order method is possible poor convergence rate. Fast iterative thresholding-like algorithms have been proposed to overcome this issue in the existing literatures. In history, another alternative way is second order algorithms or quasi-second order algorithms. In this paper, we include a quasi-Newton’s method, i.e., DFP (Davidon-Fletcher-Powell) formulations in the framework of iterative thresholding-like algorithms to replace gradient descent to form a hybrid method to further increase convergence rate. The proposed method has been performed on two numerical examples and a real-life application in sparse-spike seismic deconvolution. The numerical examples and real-life application showed that it provides an effective alternative method to solve sparsity-regularized linear inverse problems.

DOA estimation of non‐circular signals based on iterative soft thresholding algorithm

Based on compressed sensing technology, a DOA estimation algorithm for reconstructing the original signal is proposed. First, the over-complete dictionary of signal vectors is established based on the sparsity of signals in spatial domain. The DOA estimation problem is transformed into sparse signals reconstruction problem, and signals in spatial domain are reconstructed using iterative soft thresholding (IST) algorithm. At the same time, combined with the second-order statistical characteristic of non-circular signal, the array is virtual extended, and the spectrum estimation is performed by MUSIC algorithm. In addition, in order to further distinguish coherent signals, spatial smoothing technology is added to the algorithm. The simulation results show that the proposed algorithm can effectively resolve incoherent signals and coherent signals under small snapshots and low SNR.

Complex-Valued Sparse SAR-Image-Based Target Detection and Classification

It is known that synthetic aperture radar (SAR) images obtained by typical matched filtering (MF)-based algorithms always suffer from serious noise, sidelobes and clutter. However, the improvement in image quality means that the complexity of SAR systems will increase, which affects the applications of SAR images. The introduction of sparse signal processing technologies into SAR imaging proposes a new way to solve this problem. Sparse SAR images obtained by sparse recovery algorithms show better image performance than typical complex SAR images with lower sidelobes and higher signal-to-noise ratios (SNR). As the most widely applied fields of SAR images, target detection and target classification rely on SAR images with high quality. Therefore, in this paper, a target detection framework based on sparse images recovered by complex approximate message passing (CAMP) algorithm and a novel classification network via sparse images reconstructed by the new iterative soft thresholding (BiIST) algorithm are proposed. Experimental results show that sparse SAR images have better performance whether for target classification or for target detection than the images recovered by MF-based algorithms, which validates the huge application potentials of sparse images.

Advanced direction of arrival estimation using step‐learnt iterative soft‐thresholding for frequency‐modulated continuous wave multiple‐input multiple‐output radar

The number of antennas in automotive frequency-modulated continuous wave (FMCW) multiple-input multiple-output (MIMO) radar systems is increasing. Existing greedy or subspace-based methods cannot quickly and accurately estimate the direction of arrival (DoA) of the target. Therefore, we propose a fast and accurate DoA estimation algorithm for the automotive FMCW MIMO radar. To achieve both fastness and accuracy, we exploit the group sparsity in DoA estimation by defining the problem as a multiple measurement vector (MMV) compressive sensing and extend the step-learnt iterative soft-thresholding algorithm (SLISTA) to the MMV problem. To apply the extended SLISTA, we train the network in an unsupervised manner and normalise the input. We conduct experiments to evaluate the performance of the proposed method. Compared to the algorithms such as ISTA/FISTA/MFOCUSS that solve the same optimisation problem, the extended SLISTA exhibits the most accurate DoA estimation results for actual targets, with less execution time than a subspace-based method. Moreover, the results show that the extended SLISTA prevents false detections, whereas greedy and subspace-based methods do not.

Accelerating Dynamic MRI Reconstruction Using Adaptive Sequentially Truncated Higher-Order Singular Value Decomposition.

Dynamic magnetic resonance imaging (dMRI) plays an important role in cardiac perfusion and functional clinical exams. However, further applications are limited by the speed of data acquisition. A low-rank plus sparse decomposition approach is often introduced for reconstructing dynamic magnetic resonance imaging (dMRI) from highly under-sampling K-space data. In this paper, the reconstruction problem of DMR is transformed into a low-rank tensor plus sparse tensor recovery problem. A sequentially truncated higher-order singular value decomposition method is proposed to quickly approximate the low-rank tensor space structure and learn sparse components by adding a tensor kernel norm to the low-rank tensor and a l1 norm to the sparse tensor to constrain the two parts at the same time. The optimization problem is solved by using the iterative soft-thresholding algorithm; therefore, under the premise of ensuring the accuracy of the data, the amount of computation can be effectively reduced. Compared with the state-of-the-art methods, the experimental results show that the proposed method can achieve better performance in terms of reconstruction speed and reconstruction quality on 3D and 4D dMRI datasets. The multidimensional MRI time series is represented by the tensor tool and decomposed into low rank tensor terms and sparse tensor terms. The low rank spatial structure is captured by the adaptive ST-HOSVD for fast approximation and the sparse component is constrained efficiently with a sparsity transform and l1 norm. The optimization problem is solved by an iterative soft-thresholding algorithm. Through extensive 3D and 4D dMRI experiments, it is demonstrated that our method can achieve superior reconstruction performance and efficiency compared with the other three state-of-theart methods reported in the literature.

Damage localization with Lamb waves using dense convolutional sparse coding network

Composite materials are progressively employed in many safety-critical structural applications due to their superior properties. Structural health monitoring techniques based on Lamb waves have been utilized to assess the damages of composite structures. Recently, deep learning algorithms are adopted for damage detection and localization. Identifying valid damage-related features through neural networks is a crucial step in the analysis process. However, most implemented deep learning architectures are still lacking physical interpretability to some extent. In this paper, a dense convolutional sparse coding network (DCSCNet) is presented for Lamb wave-based damage localization in composite structures, providing a possibility to interpret current networks. In DCSCNet, narrowband Hanning windowed toneburst signals are utilized as kernels of the first convolutional layer to learn more meaningful features. Dense connection is theoretically demonstrated in the scope of DCSCNet, which gathers multiple feature maps directly to develop the potential of the network through feature reuse. The multi-layer iterative soft thresholding algorithm with the dense connection is then employed for solving the multi-layer convolutional sparse coding model. Moreover, effective Squeeze-Excitation is introduced as the channel attention module to boost the representational capability of the network. The experimental results demonstrate the high-performance and interpretable characteristics of the proposed DCSCNet, verifying its feasibility and effectiveness in Lamb wave-based damage localization of composite structures.

MRCON-Net: Multiscale reweighted convolutional coding neural network for low-dose CT imaging

Background and objectiveLow-dose computed tomography (LDCT) has become increasingly important for alleviating X-ray radiation damage. However, reducing the administered radiation dose may lead to degraded CT images with amplified mottle noise and nonstationary streak artifacts. Previous studies have confirmed that deep learning (DL) is promising for improving LDCT imaging. However, most DL-based frameworks are built intuitively, lack interpretability, and suffer from image detail information loss, which has become a general challenging issue. MethodsA multiscale reweighted convolutional coding neural network (MRCON-Net) is developed to address the above problems. MRCON-Net is compact and more explainable than other networks. First, inspired by the learning-based reweighted iterative soft thresholding algorithm (ISTA), we extend traditional convolutional sparse coding (CSC) to its reweighted convolutional learning form. Second, we use dilated convolution to extract multiscale image features, allowing our single model to capture the correlations between features of different scales. Finally, to automatically adjust the elements in the feature code to correct the obtained solution, a channel attention (CA) mechanism is utilized to learn appropriate weights. ResultsThe visual results obtained based on the American Association of Physicians in Medicine (AAPM) Challenge and United Image Healthcare (UIH) clinical datasets confirm that the proposed model significantly reduces serious artifact noise while retaining the desired structures. Quantitative results show that the average structural similarity index measurement (SSIM) and peak signal-to-noise ratio (PSNR) achieved on the AAPM Challenge dataset are 0.9491 and 40.66, respectively, and the SSIM and PSNR achieved on the UIH clinical dataset are 0.915 and 42.44, respectively; these are promising quantitative results. ConclusionCompared with recent state-of-the-art methods, the proposed model achieves subtle structure-enhanced LDCT imaging. In addition, through ablation studies, the components of the proposed model are validated to achieve performance improvements.