Iterative Thresholding Algorithm Research Articles

Iterative Thresholding and Projection Algorithms and Model-Based Deep Neural Networks for Sparse LQR Control Design

Iterative Thresholding and Projection Algorithms and Model-Based Deep Neural Networks for Sparse LQR Control Design

Self-Supervised Medical Image Denoising Based on WISTA-Net for Human Healthcare in Metaverse.

Medical image processing plays an important role in the interaction of real world and metaverse for healthcare. Self-supervised denoising based on sparse coding methods, without any prerequisite on large-scale training samples, has been attracting extensive attention for medical image processing. Whereas, existing self-supervised methods suffer from poor performance and low efficiency. In this paper, to achieve state-of-the-art denoising performance on the one hand, we present a self-supervised sparse coding method, named the weighted iterative shrinkage thresholding algorithm (WISTA). It does not rely on noisy-clean ground-truth image pairs to learn from only a single noisy image. On the other hand, to further improve denoising efficiency, we unfold the WISTA to construct a deep neural network (DNN) structured WISTA, named WISTA-Net. Specifically, in WISTA, motivated by the merit of the lp-norm, WISTA-Net has better denoising performance than the classical orthogonal matching pursuit (OMP) algorithm and the ISTA. Moreover, leveraging the high-efficiency of DNN structure in parameter updating, WISTA-Net outperforms the compared methods in denoising efficiency. In detail, for a 256 by 256 noisy image, the running time of WISTA-Net is 4.72 s on the CPU, which is much faster than WISTA, OMP, and ISTA by 32.88 s, 13.06 s, and 6.17 s, respectively.

Lasso-based state estimation for cyber-physical systems under sensor attacks

The development of algorithms for secure state estimation in vulnerable cyber-physical systems has been gaining attention in the last years. A consolidated assumption is that an adversary can tamper a relatively small number of sensors. In the literature, block-sparsity methods exploit this prior information to recover the attack locations and the state of the system. In this paper, we propose an alternative, Lasso-based approach and we analyse its effectiveness. In particular, we theoretically derive conditions that guarantee successful attack/state recovery, independently of established time sparsity patterns. Furthermore, we develop a sparse state observer, by starting from the iterative soft thresholding algorithm for Lasso, to perform online estimation. Through several numerical experiments, we compare the proposed methods to the state-of-the-art algorithms.

Iterative singular tube hard thresholding algorithms for tensor recovery

Iterative singular tube hard thresholding algorithms for tensor recovery

Algorithm unfolding for block-sparse and MMV problems with reduced training overhead

In this study, we consider algorithm unfolding for the multiple measurement vector (MMV) problem in the case where only few training samples are available. Algorithm unfolding has been shown to empirically speed-up in a data-driven way the convergence of various classical iterative algorithms, but for supervised learning, it is important to achieve this with minimal training data. For this, we consider learned block iterative shrinkage thresholding algorithm (LBISTA) under different training strategies. To approach almost data-free optimization at minimal training overhead, the number of trainable parameters for algorithm unfolding has to be substantially reduced. We therefore explicitly propose a reduced-size network architecture based on the Kronecker structure imposed by the MMV observation model and present the corresponding theory in this context. To ensure proper generalization, we then extend the analytic weight approach by Liu and Chen to LBISTA and the MMV setting. Rigorous theoretical guarantees and convergence results are stated for this case. We show that the network weights can be computed by solving an explicit equation at the reduced MMV dimensions which also admits a closed-form solution. Toward more practical problems, we then considered convolutional observation models and show that the proposed architecture and the analytical weight computation can be further simplified and thus open new directions for convolutional neural networks. Finally, we evaluate the unfolded algorithms in numerical experiments and discuss connections to other sparse recovering algorithms.

Gravity inversion using L0 norm for sparse constraints

SUMMARY Gravity surveys constitute an important method for investigating the Earth's interior based on density contrasts related to Earth material differentials. Because lithology depends on the environment and the period of formation, there are generally clear boundaries between rocks with different lithologies. Inversions with convex functions for approximating the L0 norm are used to detect boundaries in reconstructed models. Optimizations can easily be found because of the convex transformations; however, the volume of the reconstructed model depends on the weighting parameter and the density constraint rather than the model sparsity. To determine and adapt the modelling size, a novel non-convex framework for gravity inversion is proposed. The proposed optimization aims to directly reduce the L0 norm of the density matrix. An improved iterative hard thresholding algorithm is developed to linearly reduce the L0 penalty during the inner iteration. Accordingly, it is possible to determine the modelling scale during the iteration and achieve an expected scale for the reconstructed model. Both simple and complex model experiments demonstrate that the proposed method efficiently reconstructs models. In addition, granites formed during the Yanshanian and Indosinian periods in the Nanling region, China, are reconstructed according to the modelling size evaluated in agreement with the magnetotelluric profile and density statistics of rock samples. The known ores occur at the contact zones between the sedimentary rocks and the reconstructed Yanshanian granites. The ore-forming bodies, periods, and processes are identified, providing guidance for further deep resource exploration in the study area.

Discretized tensor-based model of total focusing method: A sparse regularization approach for enhanced ultrasonic phased array imaging

The total focusing method (TFM) is considered as the standard in ultrasonic phased array imaging and plays a vital role in industrial non-destructive testing (NDT). By utilizing the full matrix capture (FMC) dataset, the TFM can focus on every point within the specified imaging region, and is more accurate than the traditional ultrasonic phased array imaging methods. However, the TFM is essentially a delay and sum technique that often operates linearly on the time-domain signals and takes no prior information into account, so its image quality remains inadequate when dealing with defects in close proximity or scattering materials. To address this problem, this paper formulates the imaging principle of the TFM as a Boolean matrix and establishes the discretized tensor-based model. Subsequently, the model is addressed by employing the sparse regularization strategy, taking into some characteristics of industrial NDT. Regarding the solution algorithm, due to the generation of negative values by the fast iterative shrinkage threshold algorithm (FISTA), this paper introduces the rectified linear unit (ReLU) function as a non-negative constraint and presents a dedicated solution algorithm (ReLU-FISTA) for acquiring detection results. Through verification of simulation and experiment, the proposed approach exhibits superior capabilities of defect characterization and noise suppression when compared to the TFM, leading to substantial enhancements in image quality.

Terahertz wave imaging using time domain iterative shrink threshold method

We propose a new linear imaging method that combines the iterative shrinkage thresholding algorithm (ISTA) with the synthetic aperture (SA) method to achieve high resolution. Unlike other ISTA methods that use a single frequency component of signals, we use time domain signals directly as measurement, which allows for simultaneous wide bandwidth calculation and potentially better imaging results. Additionally, we simplify the backward estimation process by using impulse back-projection based on the SA method, which reduces the computing cost compared to the ISTA method. Numerical simulation and experimental measurement results show that the image resolution is similar to that of our previous non-linear imaging method, and two reflectors with different reflectivity are imaged linearly. These comparison results demonstrate the effectiveness of our proposed method in achieving high-resolution and maintaining the linearity of target reflectivity.

Application of Sparse Regularization in Spherical Radial Basis Functions-Based Regional Geoid Modeling in Colorado

Spherical radial basis function (SRBF) is an effective method for calculating regional gravity field models. Calculating gravity field models with high accuracy and resolution requires dense basis functions, resulting in complex models. This study investigated the application of sparse regularization in SRBFs-based regional gravity field modeling. L1-norm regularization, also known as the least absolute shrinkage selection operator (LASSO), was employed in the parameter estimation procedure. LASSO differs from L2-norm regularization in that the solution obtained by LASSO is sparse, specifically with a portion of the parameters being zero. A sparse model would be advantageous for improving the numerical efficiency by reducing the number of SRBFs. The optimization problem of the LASSO was solved using the fast iterative shrinkage threshold algorithm, which is known for its high efficiency. The regularization parameter was selected using the Akaike information criterion. It was specifically tailored to the L1-norm regularization problem. An approximate covariance matrix of the estimated parameters in the sparse solution was analytically constructed from a Bayesian viewpoint. Based on the remove–compute–restore technique, a regional geoid model of Colorado (USA) was calculated. The numerical results suggest that the LASSO adopted in this study provided competitive results compared to Tikhonov regularization; however, the number of basis functions in the final model was less than 25% of the Tikhonov regularization. Without significantly reducing model accuracy, the LASSO solution provides a very simple model. This is the first study to apply the LASSO to SRBFs-based modeling of the regional gravity field in real gravity observation data.

Frequency analysis of passive sonar signals using multiple-measurement physics-informed neural network

We propose a deep learning based frequency analysis for passive sonar signals by considering a linear system based on a relation between time and frequency domain signal representations. An adaptive learned iterative shrinkage thresholding algorithm (Ada-LISTA) is utilized as the deep learning architecture to solve the linear system effectively and to ensure generalization performance. To obtain more reliable solutions, we involve the linear system in loss function during training. Futhermore, in order to reduce noise effectively, we employ multiple measurements from time and space domains, which have common frequency components of passive sonar signals. Thus, the loss function is modified for solutions from the multiple measurements to have shared frequency components. We conduct experiments using synthetic and underwater in-situ data to examine performance of frequency analysis considering the linear system in designing the architecture and finding the optimal weights. The frequency analysis demonstrates superior performance in detecting frequency components and reducing noise, compared to fast Fourier transform and sparse Bayesian learning with low computational burden during its application to the test measurements.

Efficient indoor localization with wireless sensor networks via learnable sparse coding

In this study, the indoor localization problem for human movement is built as a sparse representation, and a learnable sparse coding algorithm is proposed by introducing a nonconvex penalty function as the sparse constraints. The good properties of the penalty are able to help reduce the iteration cost to find a sparse solution indicating the target locations. The sparse code can achieve its optimal value from the trained neural network model with parametrization. It is instead of a piece of algorithmic processing in the conventional methods requiring a large iteration. A real-world experiment using a Bluetooth low energy wireless sensor network validates the proposed method and shows the improvement of the error reduction at the first five iteration points on average compared to iterative shrinkage and thresholding algorithm (ISTA), and achieves competitive performance in the localization tasks for learned-ISTA-based algorithms.

TEFISTA-Net: A learnable method for high-resolution range profile reconstruction with low-frequency ultra-wideband radar

In low-frequency ultra-wideband (LFW) radar, the reconstruction of high-resolution range profile (HRRP) is an important task. The state-of-the-art compressive sensing (CS) method for HRRP reconstruction based on geometrical theory of diffraction (GTD) requires manual algorithm parameter tuning and is computationally expensive. In this paper, we propose a deep learning-based CS method for LFW radar HRRP reconstruction. We design a neural network architecture, i.e., target enhancement-based FISTA-Net (TEFISTA-Net), by unrolling the fast iterative shrinkage thresholding algorithm (FISTA). A new loss function based on the target-to-background ratio (TBR) is introduced for network training with the target enhancement capability in low signal-to-noise ratio (SNR) scenarios. The algorithm parameters in the traditional CS methods are substituted by network parameters learned from training data, getting rid of manual parameter tuning. In addition, simple convolution operations in our new method lead to lower computational complexity compared with existing methods. Experimental results based on diverse data show that the proposed method leads to higher computational efficiency with similar or better performance in comparison with existing state-of-the-art methods.

An Iterative Threshold Algorithm of Log-Sum Regularization for Sparse Problem

The log-sum function as a penalty has always been drawing widespread attention in the field of sparse problems. However, it brings a non-convex, non-smooth and non-Lipschitz optimization problem that is difficult to tackle. To overcome the problem, an iterative threshold algorithm for the sparse optimization problems with log-sum function is proposed in this paper. For brevity, the sparse optimization problems with log-sum function are named log-sum regularization. Firstly, by introducing an intermediate function to construct another new function, a property theorem about solution for log-sum regularization is established. Secondly, based on the above theorem, the optimal setting rules of the compromising parameters are elaborated, and an iterative log-sum threshold algorithm is proposed. Thirdly, under the situation that the compromising parameters of log-sum regularization are relatively small, it can be proven that the proposed algorithm converges to a local minimizer of log-sum regularization. Finally, a series of simulations are implemented to examine performance of the proposed algorithm, and the results exhibit that the proposed algorithm outperforms the state-of-the-art algorithms.

Nonsparse SAR Scene Imaging Network Based on Sparse Representation and Approximate Observations

Sparse-representation-based synthetic aperture radar (SAR) imaging technology has shown superior potential in the reconstruction of nonsparse scenes. However, many existing compressed sensing (CS) methods with sparse representation cannot obtain an optimal sparse basis and only apply to the sensing matrix obtained by exact observation, resulting in a low image quality occupying more storage space. To reduce the computational cost and improve the imaging performance of nonsparse scenes, we formulate a deep learning SAR imaging method based on sparse representation and approximated observation deduced from the chirp-scaling algorithm (CSA). First, we incorporate the CSA-derived approximated observation model and a nonlinear transform function within a sparse reconstruction framework. Second, an iterative shrinkage threshold algorithm is adopted to solve this framework, and the solving process is unfolded as a deep SAR imaging network. Third, a dual-path convolutional neural network (CNN) block is designed in the network to achieve the nonlinear transform, dramatically improving the sparse representation capability over conventional transform-domain-based CS methods. Last, we improve the CNN block to develop an enhanced version of the deep SAR imaging network, in which all the parameters are layer-varied and trained by supervised learning. The experiments demonstrate that our proposed two imaging networks outperform conventional CS-driven and deep-learning-based methods in terms of computing efficiency and reconstruction performance of nonsparse scenes.

Compressed Sensing Techniques Applied to Medical Images Obtained with Magnetic Resonance

The fast and reliable processing of medical images is of paramount importance to adequately generate data to feed machine learning algorithms that can prevent and diagnose health issues. Here, different compressed sensing techniques applied to magnetic resonance imaging are benchmarked as a means to reduce the acquisition time spent in the collection of data and signals that form the image. It is shown that by using these techniques, it is possible to reduce the number of signals needed and, therefore, substantially decrease the time to acquire the measurements. To this end, different algorithms are considered and compared: the iterative re-weighted least squares, the iterative soft thresholding algorithm, the iterative hard thresholding algorithm, the primal dual algorithm and the log barrier algorithm. Such algorithms have been implemented in different analysis programs that have been used to perform the reconstruction of the images, and it was found that the iterative soft thresholding algorithm gives the optimal results. It is found that the images obtained with this algorithm have lower quality than the original ones, but in any case, the quality should be good enough to distinguish each body structure and detect any health problems under an expert evaluation and/or statistical analysis.

Compressive beamforming for direction-of-arrival estimation of cyclostationary propeller noise

A compressive beamformer is proposed for direction-of-arrival estimation of cyclostationary ship-radiated propeller noise sources. The sources are recovered by iterative hard thresholding algorithm which is adapted for seeking the sparsity spatially and in cyclic frequency domain. Exploiting this double sparsity, the proposed cyclic compressive beamformer enables high-resolution localization of sources with different cyclic frequencies. Spectral averaging is suggested to reduce the adverse effect of the spatial Dirichlet kernel when there are multiple sources. Numerical examples demonstrate that the proposed scheme outperforms the well-known methods especially for low signal-to-noise ratios, numerous closely-spaced sources and few sensors. Performance of the proposed estimators are verified by explicit analytical derivations for several array and signal parameters.

Automatic Modulation Recognition Based on a New Deep K-SVD Denoising Algorithm

Automatic modulation recognition has a wide range of applications in wireless communication. To solve the problem that the previous methods convert signal modulation recognition into image recognition, leading to increased time costs and information loss, an automatic modulation recognition approach consisting of the improved deep K-SVD denoising algorithm is suggested. First, the effectiveness of the model for random sine wave denoising is demonstrated by simulation. Second, the original I/Q signals are fed into the modified deep K-SVD model for denoising, skipping the complicated image processing steps. Finally, the noise-reduced signals are input into a multi-channel convolutional long short-term neural network to complete the modulation recognition. To solve the slow convergence problem of Iterative Shrinkage Thresholding Algorithms in sparse coding, the Fast Iterative Shrinkage Thresholding Algorithm is adopted to improve the computational efficiency and obtain a better denoising effect. The experiments show that the improved model has an average recognition accuracy of 91.26% at different SNRs from -2 dB to 18 dB, which is better than the state-of-the-art modulation recognition models.

The application of iterative hard threshold algorithm based on nonlinear optimal compression sensing and electronic information technology in the field of automatic control

Abstract To improve the accuracy effect of the iterative hard threshold, an improved iterative hard threshold (IHT) method is proposed. The specific contents of this method include the principle of an IHT algorithm based on compression sensing (nonlinear optimization), weighted least squares improvement, the establishment of an IHT algorithm model based on weighted least squares improvement, and the experimental research of traditional algorithms and improved algorithms on one-dimensional signal reconstruction. The results show that the improved IRLSIHT algorithm takes 8.37, 29.63, and 30.86 s when the sampling rate is 0.2, 0.5, and 0.8, respectively, and the signal-to-noise ratio is 20.11, 27.47, and 31.82 dB, respectively. Compared with the traditional IHT algorithm, it takes a long time, which is a deficiency, but the signal-to-noise ratio is the largest, and the improved algorithm improves the accuracy. It has been proven that combining the method proposed in this article with automatic control can significantly save time and increase industrial output.

A projected gradient method for nonlinear inverse problems with 𝛼ℓ1 − 𝛽ℓ2 sparsity regularization

Abstract The non-convex α ⁢ ∥ ⋅ ∥ ℓ 1 − β ⁢ ∥ ⋅ ∥ ℓ 2 \alpha\lVert\,{\cdot}\,\rVert_{\ell_{1}}-\beta\lVert\,{\cdot}\,\rVert_{\ell_{2}} ( α ≥ β ≥ 0 \alpha\geq\beta\geq 0 ) regularization is a new approach for sparse recovery. A minimizer of the α ⁢ ∥ ⋅ ∥ ℓ 1 − β ⁢ ∥ ⋅ ∥ ℓ 2 \alpha\lVert\,{\cdot}\,\rVert_{\ell_{1}}-\beta\lVert\,{\cdot}\,\rVert_{\ell_{2}} regularized function can be computed by applying the ST-( α ⁢ ℓ 1 − β ⁢ ℓ 2 \alpha\ell_{1}-\beta\ell_{2} ) algorithm which is similar to the classical iterative soft thresholding algorithm (ISTA). Unfortunately, It is known that ISTA converges quite slowly, and a faster alternative to ISTA is the projected gradient (PG) method. Nevertheless, the current applicability of the PG method is limited to linear inverse problems. In this paper, we extend the PG method based on a surrogate function approach to nonlinear inverse problems with the α ⁢ ∥ ⋅ ∥ ℓ 1 − β ⁢ ∥ ⋅ ∥ ℓ 2 \alpha\lVert\,{\cdot}\,\rVert_{\ell_{1}}-\beta\lVert\,{\cdot}\,\rVert_{\ell_{2}} ( α ≥ β ≥ 0 \alpha\geq\beta\geq 0 ) regularization in the finite-dimensional space R n \mathbb{R}^{n} . It is shown that the presented algorithm converges subsequentially to a stationary point of a constrained Tikhonov-type functional for sparsity regularization. Numerical experiments are given in the context of a nonlinear compressive sensing problem to illustrate the efficiency of the proposed approach.

Fluorescence microscopy images denoising via deep convolutional sparse coding

Fluorescence microscopy images captured in low light and short exposure time conditions are always contaminated by photons and readout noises, which reduce the fluorescence microscopy images quality. In most cases, this kind of noise can be modeled as Poisson–Gaussian noise. Correspondingly, its denoising task has always been a hot but challenging topic in recent years. In this paper, by integrating model-driven and learning-driven methodologies, we propose an end-to-end supervised neural network for fluorescence microscopy images denoising, named MCSC-net, which embeds the multi-layer learned iterative soft threshold algorithm (ML-LISTA) into deep convolutional neural network (DCNN). Our approach not only uses the strong learning ability of DCNN to adaptively update all parameters in the ML-LISTA, but also introduces dilated convolution into network training without additional parameters to improve denoising performance. In addition, compared with several related methods on a real data set of fluorescence microscopy images, MCSC-net achieves the best denoising effects both in qualitative and quantitative aspects, which shows its strong appeal in practical denoising applications.