Iterative Hard Thresholding Algorithm Research Articles

Sparse recovery using expanders via hard thresholding algorithm

Expanders play an important role in combinatorial compressed sensing. Via expanders measurements, we propose the expander normalized heavy ball hard thresholding algorithm (ENHB-HT) based on expander iterative hard thresholding (E-IHT) algorithm. We provide convergence analysis of ENHB-HT, and it turns out that ENHB-HT can recover an s-sparse signal if the measurement matrix A∈{0,1}m×n satisfies some mild conditions. Numerical experiments are simulated to support our two main theorems which describe the convergence rate and the accuracy of the proposed algorithm. Simulations are also performed to compare the performance of ENHB-HT and several existing algorithms under different types of noise, the empirical results demonstrate that our algorithm outperform a few existing ones in the presence of outliers.

Generalized [formula omitted]-sparse algorithms for constructing fault tolerant RBF networks

In the construction process of radial basis function (RBF) networks, two common crucial issues arise: the selection of RBF centers and the effective utilization of the given source without encountering the overfitting problem. Another important issue is the fault tolerant capability. That is, when noise or faults exist in a trained network, it is crucial that the network’s performance does not undergo significant deterioration or decrease. However, without employing a fault tolerant procedure, a trained RBF network may exhibit significantly poor performance. Unfortunately, most existing algorithms are unable to simultaneously address all of the aforementioned issues. This paper proposes fault tolerant training algorithms that can simultaneously select RBF nodes and train RBF output weights. Additionally, our algorithms can directly control the number of RBF nodes in an explicit manner, eliminating the need for a time-consuming procedure to tune the regularization parameter and achieve the target RBF network size. Based on simulation results, our algorithms demonstrate improved test set performance when more RBF nodes are used, effectively utilizing the given source without encountering the overfitting problem. This paper first defines a fault tolerant objective function, which includes a term to suppress the effects of weight faults and weight noise. This term also prevents the issue of overfitting, resulting in better test set performance when more RBF nodes are utilized. With the defined objective function, the training process is designed to solve a generalized M-sparse problem by incorporating an ℓ0-norm constraint. The ℓ0-norm constraint allows us to directly and explicitly control the number of RBF nodes. To address the generalized M-sparse problem, we introduce the noise-resistant iterative hard thresholding (NR-IHT) algorithm. The convergence properties of the NR-IHT algorithm are subsequently discussed theoretically. To further enhance performance, we incorporate the momentum concept into the NR-IHT algorithm, referring to the modified version as “NR-IHT-Mom”. Simulation results show that both the NR-IHT algorithm and the NR-IHT-Mom algorithm outperform several state-of-the-art comparison algorithms.

Sufficient condition based on nearly optimal order RIC for IHT algorithm

Iterative hard thresholding (IHT) is one class of sparse signal recovery algorithms widely used in compressed sensing. The well-known restricted isometry constant (RIC) is one of the most popular frameworks used to ensure the convergence of recovery algorithms. In previous literature, sufficient conditions to guarantee the recovery performance of the IHT algorithm have usually been established based on RICs with orders 3 s and 2 s , while no such condition in terms of RIC with order less than 2 s has been reported yet. In this study, based on the theoretical optimal step-length, we develop the RIC condition γ < ( 5 − 1 ) / 8 ≈ 0.1545 for successful recovery via the IHT algorithm. Here, γ = δ s if s is an even number, otherwise γ = δ s + 1 . To the best of our knowledge, this is the first result based on RIC with nearly optimal order for the IHT algorithm.

Iterative singular tube hard thresholding algorithms for tensor recovery

Iterative singular tube hard thresholding algorithms for tensor recovery

Gravity inversion using L0 norm for sparse constraints

SUMMARY Gravity surveys constitute an important method for investigating the Earth's interior based on density contrasts related to Earth material differentials. Because lithology depends on the environment and the period of formation, there are generally clear boundaries between rocks with different lithologies. Inversions with convex functions for approximating the L0 norm are used to detect boundaries in reconstructed models. Optimizations can easily be found because of the convex transformations; however, the volume of the reconstructed model depends on the weighting parameter and the density constraint rather than the model sparsity. To determine and adapt the modelling size, a novel non-convex framework for gravity inversion is proposed. The proposed optimization aims to directly reduce the L0 norm of the density matrix. An improved iterative hard thresholding algorithm is developed to linearly reduce the L0 penalty during the inner iteration. Accordingly, it is possible to determine the modelling scale during the iteration and achieve an expected scale for the reconstructed model. Both simple and complex model experiments demonstrate that the proposed method efficiently reconstructs models. In addition, granites formed during the Yanshanian and Indosinian periods in the Nanling region, China, are reconstructed according to the modelling size evaluated in agreement with the magnetotelluric profile and density statistics of rock samples. The known ores occur at the contact zones between the sedimentary rocks and the reconstructed Yanshanian granites. The ore-forming bodies, periods, and processes are identified, providing guidance for further deep resource exploration in the study area.

A ReLU-based hard-thresholding algorithm for non-negative sparse signal recovery

In numerous applications, such as DNA microarrays, face recognition, and spectral unmixing, we need to acquire a non-negative K-sparse signal x from an underdetermined linear model y=Ax+v, where A is a sensing matrix and v is a noise vector. In this paper, we first propose a ReLU-based hard-thresholding algorithm (RHT) to recover x by taking advantage of its non-negative sparsity. Two sufficient conditions for stable recovery with RHT are then developed, which are respectively based on the restricted isometry property (RIP) and mutual coherence of the sensing matrix A. As far as we know, these two sufficient conditions are the best for hard-thresholding-type algorithms. Numerical experiments show that RHT has better overall recovery performance in the recovery non-negative sparse signals than the non-negative least squares (NNLS) algorithm, some hard-thresholding-type algorithms including the iterative hard-thresholding (IHT) algorithm, hard-thresholding pursuit (HTP), Newton-step-based iterative hard-thresholding algorithm (NSIHT), and Newton-step-based hard-thresholding pursuit (NSHTP), several non-negative sparse recovery approaches including non-negative orthogonal matching pursuit (NNOMP), fast NNOMP (FNNOMP), non-negative orthogonal least squares (NNOLS), and non-negative regularization (NNREG) method. In terms of efficiency, RHT has a similar performance to IHT, and is much more efficient than other tested algorithms.

Compressed Sensing Techniques Applied to Medical Images Obtained with Magnetic Resonance

The fast and reliable processing of medical images is of paramount importance to adequately generate data to feed machine learning algorithms that can prevent and diagnose health issues. Here, different compressed sensing techniques applied to magnetic resonance imaging are benchmarked as a means to reduce the acquisition time spent in the collection of data and signals that form the image. It is shown that by using these techniques, it is possible to reduce the number of signals needed and, therefore, substantially decrease the time to acquire the measurements. To this end, different algorithms are considered and compared: the iterative re-weighted least squares, the iterative soft thresholding algorithm, the iterative hard thresholding algorithm, the primal dual algorithm and the log barrier algorithm. Such algorithms have been implemented in different analysis programs that have been used to perform the reconstruction of the images, and it was found that the iterative soft thresholding algorithm gives the optimal results. It is found that the images obtained with this algorithm have lower quality than the original ones, but in any case, the quality should be good enough to distinguish each body structure and detect any health problems under an expert evaluation and/or statistical analysis.

Compressive beamforming for direction-of-arrival estimation of cyclostationary propeller noise

A compressive beamformer is proposed for direction-of-arrival estimation of cyclostationary ship-radiated propeller noise sources. The sources are recovered by iterative hard thresholding algorithm which is adapted for seeking the sparsity spatially and in cyclic frequency domain. Exploiting this double sparsity, the proposed cyclic compressive beamformer enables high-resolution localization of sources with different cyclic frequencies. Spectral averaging is suggested to reduce the adverse effect of the spatial Dirichlet kernel when there are multiple sources. Numerical examples demonstrate that the proposed scheme outperforms the well-known methods especially for low signal-to-noise ratios, numerous closely-spaced sources and few sensors. Performance of the proposed estimators are verified by explicit analytical derivations for several array and signal parameters.

The application of iterative hard threshold algorithm based on nonlinear optimal compression sensing and electronic information technology in the field of automatic control

Abstract To improve the accuracy effect of the iterative hard threshold, an improved iterative hard threshold (IHT) method is proposed. The specific contents of this method include the principle of an IHT algorithm based on compression sensing (nonlinear optimization), weighted least squares improvement, the establishment of an IHT algorithm model based on weighted least squares improvement, and the experimental research of traditional algorithms and improved algorithms on one-dimensional signal reconstruction. The results show that the improved IRLSIHT algorithm takes 8.37, 29.63, and 30.86 s when the sampling rate is 0.2, 0.5, and 0.8, respectively, and the signal-to-noise ratio is 20.11, 27.47, and 31.82 dB, respectively. Compared with the traditional IHT algorithm, it takes a long time, which is a deficiency, but the signal-to-noise ratio is the largest, and the improved algorithm improves the accuracy. It has been proven that combining the method proposed in this article with automatic control can significantly save time and increase industrial output.

Iterative descent group hard thresholding algorithms for block sparsity

In this paper we consider the problem of recovering block-sparse structures in a linear regression context. Penalized mean squared criteria are generally considered in such contexts where ℓ2,1 mixed norm penalty terms is often used as a convex alternative to the ℓ2,0 penalty. Here, we propose an iterative block cyclic descent algorithm approach to address the case of an ℓ2,0 penalty. We prove its convergence and illustrate its potential benefit compared to ℓ2,1 or ℓ2,q (0<q≤1) penalization. We also propose a momentum approach for accelerated convergence and an application to sensor positioning for array processing.

Photonic compressive sampling of wideband sparse radio frequency signals with 1-bit quantization.

Photonic compressive sampling (PCS) is an effective method to recover wideband sparse radio frequency (RF) signals. However, the noisy and high-loss photonic link leads to signal-to-noise ratio (SNR) degradation of the RF signal to be tested, which limits the recovery performance of the PCS system. In this paper, a random demodulator-based PCS system with 1-bit quantization is proposed. The system consists of a photonic mixer, a low-pass filter, a 1-bit analog-to-digital converter (ADC), and a digital signal processor (DSP). The 1-bit quantized result is used to recover the spectra of the wideband sparse RF signal with the binary iterative hard thresholding (BIHT) algorithm, which can alleviate the negative impact of the SNR degradation caused by the photonic link. A full theoretical framework of the PCS system with 1-bit quantization is given. Simulation results show that the PCS system with 1-bit quantization can provide better recovery performance than the traditional PCS system under low SNR and stringent bit budget.

3D-EPI blip-up/down acquisition (BUDA) with CAIPI and joint Hankel structured low-rank reconstruction for rapid distortion-free high-resolution mapping.

This work aims to develop a novel distortion-free 3D-EPI acquisition and image reconstruction technique for fast and robust, high-resolution, whole-brain imaging as well as quantitative mapping. 3D Blip-up and -down acquisition (3D-BUDA) sequence is designed for both single- and multi-echo 3D gradient recalled echo (GRE)-EPI imaging using multiple shots with blip-up and -down readouts to encode B0 field map information. Complementary k-space coverage is achieved using controlled aliasing in parallel imaging (CAIPI) sampling across the shots. For image reconstruction, an iterative hard-thresholding algorithm is employed to minimize the cost function that combines field map information informed parallel imaging with the structured low-rank constraint for multi-shot 3D-BUDA data. Extending 3D-BUDA to multi-echo imaging permits mapping. For this, we propose constructing a joint Hankel matrix along both echo and shot dimensions to improve the reconstruction. Experimental results on in vivo multi-echo data demonstrate that, by performing joint reconstruction along with both echo and shot dimensions, reconstruction accuracy is improved compared to standard 3D-BUDA reconstruction. CAIPI sampling is further shown to enhance image quality. For mapping, parameter values from 3D-Joint-CAIPI-BUDA and reference multi-echo GRE are within limits of agreement as quantified by Bland-Altman analysis. The proposed technique enables rapid 3D distortion-free high-resolution imaging and mapping. Specifically, 3D-BUDA enables 1-mm isotropic whole-brain imaging in 22 s at 3T and 9s on a 7T scanner. The combination of multi-echo 3D-BUDA with CAIPI acquisition and joint reconstruction enables distortion-free whole-brain mapping in 47 s at 1.1 × 1.1 × 1.0 mm3 resolution.

A Hybrid Iterative Thresholding Method for Solving Sparsity-Regularized Linear Inverse Problems and Its Application in Seismic Sparse-Spike Deconvolution

Sparsity-regularized linear inverse problem has been served as the base in many disciplines, such as remote sensing imaging, image processing and analysis, seismic deconvolution, compressed sensing, medical imaging, and so forth. Iterative hard thresholding algorithm (IHTA) and iterative soft thresholding algorithm (ISTA) are two frequently used methods to solve sparsity-regularized linear inverse problems. They are also the basic unit of other more complex methods. IHTA and ISTA are derived under steepest descent method. I.e., iteratively perform gradient descent and thresholding shrinkage. Steepest descent method is a first order algorithm, which is a powerful way to solve optimization due to its relatively simple implementation. However, a known issue of first order method is possible poor convergence rate. Fast iterative thresholding-like algorithms have been proposed to overcome this issue in the existing literatures. In history, another alternative way is second order algorithms or quasi-second order algorithms. In this paper, we include a quasi-Newton’s method, i.e., DFP (Davidon-Fletcher-Powell) formulations in the framework of iterative thresholding-like algorithms to replace gradient descent to form a hybrid method to further increase convergence rate. The proposed method has been performed on two numerical examples and a real-life application in sparse-spike seismic deconvolution. The numerical examples and real-life application showed that it provides an effective alternative method to solve sparsity-regularized linear inverse problems.

Improved Sufficient Conditions Based on RIC of Order 2s for IHT and HTP Algorithms

Reconstructing an s-sparse signal x from an un- derdetermined noisy linear model arises in many applications. The iterative hard thresholding (IHT) and hard thresholding pursuit (HTP) algorithms are two popular sparse signal recovery algorithms. Since their recovery performances can be theoreti- cally characterized by their sufficient conditions of stable sparse signal recovery, it is essential to establish less restrictive sufficient conditions. This work develops δ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2s</sub> -based sufficient conditions for stable recovery with IHT and HTP. The two new sufficient conditions are significantly less restrictive than the state-of-the- art ones for IHT and HTP.

Video anomaly detection by fusing self-attention and autoencoder

Video anomaly detection by fusing self-attention and autoencoder

Adaptive iterative hard thresholding for low-rank matrix recovery and rank-one measurements

In low-rank matrix recovery, many kinds of measurements fail to meet the standard restricted isometry property (RIP), such as rank-one measurements, that is, [A(X)]i=〈Ai,X〉 with rank(Ai)=1, i=1,...,m. Historical iterative hard thresholding sequence for low-rank matrix recovery and rank-one measurements was taken as Xn+1=Ps(Xn−μnPt(A⁎sign(A(Xn)−y))), which introduced the “tail” and “head” approximations Ps and Pt, respectively. In this paper, we remove the term Pt and provide a new iterative hard thresholding algorithm with adaptive step size (abbreviated as AIHT). The linear convergence analysis and stability results on AIHT are established under the ℓ1/ℓ2-RIP. Particularly, we discuss the rank-one Gaussian measurements under the tight upper and lower bounds on E‖A(X)‖1, and provide better convergence rate and sampling complexity. Besides, several empirical experiments are provided to show that AIHT performs better than the historical rank-one iterative hard thresholding method.

Iterative Hard Thresholding Algorithm-Based Detector for Compressed OFDM-IM Systems

A low-complexity iterative hard thresholding (IHT) algorithm-based detector is proposed for the compressed orthogonal frequency division multiplexing with index modulation (OFDM-IM) system. To improve recovery accuracy and reduce the number of iterations, detection is embedded into the iterations of the IHT algorithm, and the block energy information is used to identify subblocks that are more likely to be error-recovered. Compared with the existing methods for improving spectral efficiency, the new method does not expand the detection search space and not need extra pseudo-inverse calculations during iterations. Theoretical and simulation results show that the proposed scheme can considerably reduce computational complexity while improving spectral efficiency and preserving significant BER performance gain.

Stability selection enables robust learning of differential equations from limited noisy data.

We present a statistical learning framework for robust identification of differential equations from noisy spatio-temporal data. We address two issues that have so far limited the application of such methods, namely their robustness against noise and the need for manual parameter tuning, by proposing stability-based model selection to determine the level of regularization required for reproducible inference. This avoids manual parameter tuning and improves robustness against noise in the data. Our stability selection approach, termed PDE-STRIDE, can be combined with any sparsity-promoting regression method and provides an interpretable criterion for model component importance. We show that the particular combination of stability selection with the iterative hard-thresholding algorithm from compressed sensing provides a fast and robust framework for equation inference that outperforms previous approaches with respect to accuracy, amount of data required, and robustness. We illustrate the performance of PDE-STRIDE on a range of simulated benchmark problems, and we demonstrate the applicability of PDE-STRIDE on real-world data by considering purely data-driven inference of the protein interaction network for embryonic polarization in Caenorhabditis elegans. Using fluorescence microscopy images of C. elegans zygotes as input data, PDE-STRIDE is able to learn the molecular interactions of the proteins.

Reconstruction Algorithm-Based CT Imaging for the Diagnosis of Hepatic Ascites

The study was aimed at exploring the diagnostic value of artificial intelligence reconstruction algorithm combined with CT image parameters on hepatic ascites, expected to provide a reference for the etiological evaluation of clinical abdominal effusion. Specifically, the adaptive iterative hard threshold (AIHT) algorithm for CT image reconstruction was proposed. Then, 100 patients with peritoneal effusion were selected as the research subjects. After 8 cases were excluded, the remaining was divided into 50 cases of the S1 group (hepatic ascites) and 42 cases of the D0 group (cancerous peritoneal effusion). Gemstone energy spectrum CT scanning was performed on all patients, and CT image parameters of the two groups were compared. It was found that CT value of mixed energy, CT value of 60-100 KeV single energy, concentration value of water (calcium), concentration value of water (iodine), and slope of energy spectrum curve in the S1 group were significantly lower than those in the D0 group (P < 0.05). The effective atomic number in the S1 group was significantly higher than that in the D0 group (P < 0.05). Of the 50 patients in the S1 group, 3 (6%) had an ascending and 47 (94%) had a descending spectral curve. Of the 42 patients in the D0 group, 37 (88.1%) had an ascending and 5 (11.9%) had a descending spectral curve. The sensitivity and specificity of water (iodine) were 0.927 and 0.836, respectively. The sensitivity and specificity of water (calcium) were 0.863 and 0.887, respectively. For different scan ranges ([0,90]; [0,120]), root mean square error (RMSE) of AIHT reconstructed image was significantly smaller than that of traditional algorithm, while peak signal-to-noise ratio (PSNR) was opposite. The differences were statistically significant (P < 0.05). In conclusion, AIHT-based CT images can better display the distribution of hepatic ascites, and the parameters of CT value, effective atomic number, water (iodine), water (calcium), and spectral curve can all provide help for the identification of hepatic ascites. Especially, water (iodine) and water (calcium) demonstrated high diagnostic performance of hepatic ascites.

Exploiting prior knowledge in compressed sensing to design robust systems for endoscopy image recovery

In this work, we investigate compressed sensing (CS) techniques based on the exploitation of prior knowledge to support telemedicine. In particular, prior knowledge is obtained by computing the probability of appearance of non-zero elements in each row of a sparse matrix, which is then employed in sensing matrix design and recovery algorithms for CS systems. A robust sensing matrix is designed by jointly reducing the average mutual coherence and the projection of the sparse representation error. A Probability-Driven Normalized Iterative Hard Thresholding (PD-NIHT) algorithm is developed as the recovery method, which also exploits the prior knowledge of the probability of appearance of non-zero elements and can bring performance benefits. Simulations for synthetic data and different organs of endoscopy image are carried out, where the proposed sensing matrix and PD-NIHT algorithm achieve a better performance than previously reported algorithms.