Iterative Shrinkage Thresholding Research Articles

Improved Analytic Learned Iterative Shrinkage Thresholding Algorithm and Its Application to Tomographic Synthetic Aperture Radar Building Object Height Inversion

Tomographic Synthetic Aperture Radar (TomoSAR) building object height inversion is a sparse reconstruction problem that utilizes the data obtained from several spacecraft passes to invert the scatterer position in the height direction. In practical applications, the number of passes is often small, and the observation data are also small due to the objective conditions, so this study focuses on the inversion under the restricted observation data conditions. The Analytic Learned Iterative Shrinkage Thresholding Algorithm (ALISTA) is a kind of deep unfolding network algorithm, which is a combination of the Iterative Shrinkage Thresholding Algorithm (ISTA) and deep learning technology, and it has the advantages of both. The ALISTA is one of the representative algorithms for TomoSAR building object height inversion. However, the structure of the ALISTA algorithm is simple, which has neither the excellent connection structure of a deep learning network nor the acceleration format combined with the ISTA algorithm. Therefore, this study proposes two directions of improvement for the ALISTA algorithm: firstly, an improvement in the inter-layer connection of the network by introducing a connection similar to residual networks obtains the Extragradient Analytic Learned Iterative Shrinkage Thresholding Algorithm (EALISTA) and further proves that the EALISTA achieves linear convergence; secondly, there is an improvement in the iterative format of the intra-layer iteration of the network by introducing the Nesterov momentum acceleration, which obtains the Fast Analytic Learned Iterative Shrinkage Thresholding Algorithm (FALISTA). We first performed inversion experiments on simulated data, which verified the effectiveness of the two proposed algorithms. Then, we conducted TomoSAR building object height inversion experiments on limited measured data and used the deviation metric P to measure the robustness of the algorithms to invert under restricted observation data. The results show that both proposed algorithms have better robustness, which verifies the superior performance of the two algorithms. In addition, we further analyze how to choose the most suitable algorithms for inversion in engineering practice applications based on the results of the experiments on measured data.

Cipher-image compression by using compressive sensing and interweaving permutation

Recently, compressing cipher images by using compressive sensing (CS) has attracted a lot of attention. However, the previous CS-based works cannot provide satisfactory compression performance. In this article, a cipher-image compression scheme based on CS and interweaving permutation is proposed to address this concern. Before image compression, the nature image is down-sampled to generate a lot of patches by using interweaving permutation firstly and then each patch is encrypted by a scrambling matrix, where each patch is a down-sampled image. In this way, the encrypted patches are highly correlated from one to each other since they preserve the correlation of the original patches. Therefore, we can encode the cipher image by employing CS and differential pulse-code modulation (DPCM) in an efficient way. To reconstruct the image effectively, an iterative shrinkage-thresholding (IST) algorithm is developed. Compared with the previous CS-based works, our scheme achieves better ratio-distortion (R-D) performance.

Learned Two-Step Iterative Shrinkage Thresholding Algorithm for Deep Compressive Sensing

Learned Two-Step Iterative Shrinkage Thresholding Algorithm for Deep Compressive Sensing

CTV-Net: Complex-Valued TV-Driven Network With Nested Topology for 3-D SAR Imaging.

The regularization-based approaches offer promise in improving synthetic aperture radar (SAR) imaging quality while reducing system complexity. However, the widely applied l1 regularization model is hindered by their hypothesis of inherent sparsity, causing unreal estimations of surface-like targets. Inspired by the edge-preserving property of total variation (TV), we propose a new complex-valued TV (CTV)-driven interpretable neural network with nested topology, i.e., CTV-Net, for 3-D SAR imaging. In our scheme, based on the 2-D holography imaging operator, the CTV-driven optimization model is constructed to pursue precise estimations in weakly sparse scenarios. Subsequently, a nested algorithmic framework, i.e., complex-valued TV-driven fast iterative shrinkage thresholding (CTV-FIST), is derived from the theory of proximal gradient descent (PGD) and FIST algorithm, theoretically supporting the design of CTV-Net. In CTV-Net, the trainable weights are layer-varied and functionally relevant to the hyperparameters of CTV-FIST, which aims to constrain the algorithmic parameters to update in a well-conditioned tendency. All weights are learned by end-to-end training based on a two-term cost function, which bounds the measurement fidelity and TV norm simultaneously. Under the guidance of the SAR signal model, a reasonably sized training set is generated, by randomly selecting reference images from the MNIST set and consequently synthesizing complex-valued label signals. Finally, the methodology is validated, numerically and visually, by extensive SAR simulations and real-measured experiments, and the results demonstrate the viability and efficiency of the proposed CTV-Net in the cases of recovering 3-D SAR images from incomplete echoes.

Migration through Resolution Cell Correction and Sparse Aperture ISAR Imaging for Maneuvering Target Based on Whale Optimization Algorithm-Fast Iterative Shrinkage Thresholding Algorithm.

Targets faced by inverse synthetic aperture radar (ISAR) are often non-cooperative, with target maneuvering being the main manifestation of this non-cooperation. Maneuvers cause ISAR imaging results to be severely defocused, which can create huge difficulties in target identification. In addition, as the ISAR bandwidth continues to increase, the impact of migration through resolution cells (MTRC) on imaging results becomes more significant. Target non-cooperation may also result in sparse aperture, leading to the failure of traditional ISAR imaging algorithms. Therefore, this paper proposes an algorithm to realize MTRC correction and sparse aperture ISAR imaging for maneuvering targets simultaneously named whale optimization algorithm-fast iterative shrinkage thresholding algorithm (WOA-FISTA). In this algorithm, FISTA is used to perform MTRC correction and sparse aperture ISAR imaging efficiently and WOA is adopted to estimate the rotational parameter to eliminate the effects of maneuvering on imaging results. Experimental results based on simulation and measured datasets prove that the proposed algorithm implements sparse aperture ISAR imaging and MTRC correction for maneuvering targets simultaneously. The proposed algorithm achieves better results than traditional algorithms under different signal-to-noise ratio conditions.

EIDNet: Extragradient-based iterative denoising network for image compressive sensing reconstruction

Compressive sensing (CS) is a kind of hardware-friendly technology in effective image reconstruction. Most of the existing CS reconstruction methods can be classified into model-driven iterative optimization methods and data-driven neural network methods. These two types of methods have their own problems with computational complexity and theoretical interpretability. Motivated by the deep unfolding theory, we develop the model-driven Extragradient-based Iterative Denoising Network, dubbed EIDNet, which is based on the denoising model of the iterative shrinkage threshold algorithm (ISTA). By unfolding the iterative denoising model into the deep learning network, EIDNet addresses the interpretability problem of neural network methods in CS reconstruction. Each denoising iteration of the optimization algorithm is mapped into one layer of the reconstruction network in EIDNet. The extragradient method is adopted to accelerate network convergence. In addition, we introduce a learning-based sampling and initialization network for adaptive sampling and initialization. Extensive experiments show that the proposed EIDNet can obtain better image CS reconstruction performance than other state-of-the-art CS methods while maintaining satisfactory reconstruction efficiency.

Robust treatment planning for small animal radio-neuromodulation using focused kV x-ray beams.

In preclinical radio-neuromodulation research, small animal experiments are pivotal for unraveling radiobiological mechanism, investigating prescription and planning techniques, and assessing treatment effects and toxicities. However, the target size inside a rat brain is typically in the order of sub-millimeters. The small target inside the visual cortex neural region in rat brain with a diameter of around 1mm was focused in this work to observe the physiological change of this region. Delivering uniform doses to the small target while sparing health tissues is challenging. Focused kV x-ray technique based on modern x-ray polycapillary focusing lens is a promising modality for small animal radio-neuromodulation. The current manual planning method could lead to sub-optimal plans, and the positioning uncertainties due to mechanical accuracy limitations, animal immobilization, and robotic arm motion are not considered. This work aims to design a robust inverse planning method to optimize the intensities of focused kV x-ray beams located in beam trajectories to irradiate small mm-sized targets in rat brains for radio-neuromodulation. Focused kV x-ray beams were generated through polycapillary x-ray focusing lenses on achieving small (≤0.3mm) focus perpendicular to the beam. The beam trajectories were manually designed in 3D space in scanning-while-rotating mode. Geant4 Monte Carlo (MC) simulation generated a dose calculation matrix for each focused kV x-ray beam located in beam trajectories. In the proposed robust inverse planning method, an objective function combining a voxel-wise stochastic programming approach and L1 norm regularization was established to overcome the positioning uncertainties and obtain a high-quality plan. The fast iterative shrinkage thresholding algorithm (FISTA) was utilized to solve the objective function and obtain the optimal intensities. Four cases were employed to validate the feasibility and effectiveness of the proposed method. The manual and non-robust inverse planning methods were also implemented for comparison. The proposed robust inverse planning method achieved superior dose homogeneity and higher robustness against positioning uncertainties. On average, the clinical target volume (CTV) homogeneity index (HI) of robust inverse plan improved to 13.3 from 22.9 in non-robust inverse plan and 53.8 in manual plan if positioning uncertainties were also present. The average bandwidth at D90 was reduced by 6.5Gy in the robust inverse plan, compared to 9.6Gy in non-robust inverse plan and 12.5Gy in manual plan. The average bandwidth at D80 was reduced by 3.4Gy in robust inverse plan, compared to 5.5Gy in non-robust inverse plan and 8.5Gy in manual plan. Moreover, the dose delivery time of manual plan was reduced by an average reduction of 54.7% with robust inverse plan and 29.0% with non-robust inverse plan. Compared to manual and non-robust inverse planning methods, the robust inverse planning method improved the dose homogeneity and delivery efficiency and was resistant to the uncertainties, which are crucial for radio-neuromodulation utilizing focused kV x-rays.

SWISTA-Nets: Subband-adaptive wavelet iterative shrinkage thresholding networks for image reconstruction

Robust and interpretable image reconstruction is central to imageology applications in clinical practice. Prevalent deep networks, with strong learning ability to extract implicit information from data manifold, are still lack of prior knowledge introduced from mathematics or physics, leading to instability, poor structure interpretability and high computation cost. As to this issue, we propose two prior knowledge-driven networks to combine the good interpretability of mathematical methods and the powerful learnability of deep learning methods. Incorporating different kinds of prior knowledge, we propose subband-adaptive wavelet iterative shrinkage thresholding networks (SWISTA-Nets), where almost every network module is in one-to-one correspondence with each step involved in the iterative algorithm. By end-to-end training of proposed SWISTA-Nets, implicit information can be extracted from training data and guide the tuning process of key parameters that possess mathematical definition. The inverse problems associated with two medical imaging modalities, i.e., electromagnetic tomography and X-ray computational tomography are applied to validate the proposed networks. Both visual and quantitative results indicate that the SWISTA-Nets outperform mathematical methods and state-of-the-art prior knowledge-driven networks, especially with fewer training parameters, interpretable network structures and well robustness. We assume that our analysis will support further investigation of prior knowledge-driven networks in the field of ill-posed image reconstruction.

A Fast Intersection of Confidence Intervals Method-Based Adaptive Thresholding for Sparse Image Reconstruction Using the Matrix Form of the Wavelet Transform

One of the frequently used classes of sparse reconstruction algorithms is based on the iterative shrinkage/thresholding procedure, in which the thresholding parameter controls a trade-off between the algorithm’s accuracy and execution time. In order to avoid this trade-off, we propose using a fast intersection of confidence intervals method in order to adaptively control the threshold value throughout the iterations of the reconstruction algorithm. We have upgraded the two-step iterative shrinkage thresholding algorithm with a such procedure, improving its performance. The proposed algorithm, denoted as the FICI-TwIST, along with a few selected state-of-the-art sparse reconstruction algorithms, has been tested on the classical problem of image recovery by emphasizing the image sparsity in the discrete cosine and the discrete wavelet domain. Furthermore, we have derived a single wavelet transformation matrix which avoids wrapping effects, thereby achieving significantly faster execution times as compared to a more traditional function-based transformation. The obtained results indicate the competitive performance of the proposed algorithm, even in cases where all algorithm parameters have been individually fine-tuned for best performance.

MsDC-DEQ-Net: Deep Equilibrium Model (DEQ) with Multiscale Dilated Convolution for Image Compressive Sensing (CS)

Compressive sensing (CS) is a technique that enables the recovery of sparse signals using fewer measurements than traditional sampling methods. To address the computational challenges of CS reconstruction, our objective is to develop an interpretable and concise neural network model for reconstructing natural images using CS. We achieve this by mapping one step of the iterative shrinkage thresholding algorithm (ISTA) to a deep network block, representing one iteration of ISTA. To enhance learning ability and incorporate structural diversity, we integrate aggregated residual transformations (ResNeXt) and squeeze-and-excitation mechanisms into the ISTA block. This block serves as a deep equilibrium layer connected to a semi-tensor product network for convenient sampling and providing an initial reconstruction. The resulting model, called MsDC-DEQ-Net, exhibits competitive performance compared to state-of-the-art network-based methods. It significantly reduces storage requirements compared to deep unrolling methods, using only one iteration block instead of multiple iterations. Unlike deep unrolling models, MsDC-DEQ-Net can be iteratively used, gradually improving reconstruction accuracy while considering computation tradeoffs. Additionally, the model benefits from multiscale dilated convolutions, further enhancing performance.

Self-Supervised Medical Image Denoising Based on WISTA-Net for Human Healthcare in Metaverse.

Medical image processing plays an important role in the interaction of real world and metaverse for healthcare. Self-supervised denoising based on sparse coding methods, without any prerequisite on large-scale training samples, has been attracting extensive attention for medical image processing. Whereas, existing self-supervised methods suffer from poor performance and low efficiency. In this paper, to achieve state-of-the-art denoising performance on the one hand, we present a self-supervised sparse coding method, named the weighted iterative shrinkage thresholding algorithm (WISTA). It does not rely on noisy-clean ground-truth image pairs to learn from only a single noisy image. On the other hand, to further improve denoising efficiency, we unfold the WISTA to construct a deep neural network (DNN) structured WISTA, named WISTA-Net. Specifically, in WISTA, motivated by the merit of the lp-norm, WISTA-Net has better denoising performance than the classical orthogonal matching pursuit (OMP) algorithm and the ISTA. Moreover, leveraging the high-efficiency of DNN structure in parameter updating, WISTA-Net outperforms the compared methods in denoising efficiency. In detail, for a 256 by 256 noisy image, the running time of WISTA-Net is 4.72 s on the CPU, which is much faster than WISTA, OMP, and ISTA by 32.88 s, 13.06 s, and 6.17 s, respectively.

Algorithm unfolding for block-sparse and MMV problems with reduced training overhead

In this study, we consider algorithm unfolding for the multiple measurement vector (MMV) problem in the case where only few training samples are available. Algorithm unfolding has been shown to empirically speed-up in a data-driven way the convergence of various classical iterative algorithms, but for supervised learning, it is important to achieve this with minimal training data. For this, we consider learned block iterative shrinkage thresholding algorithm (LBISTA) under different training strategies. To approach almost data-free optimization at minimal training overhead, the number of trainable parameters for algorithm unfolding has to be substantially reduced. We therefore explicitly propose a reduced-size network architecture based on the Kronecker structure imposed by the MMV observation model and present the corresponding theory in this context. To ensure proper generalization, we then extend the analytic weight approach by Liu and Chen to LBISTA and the MMV setting. Rigorous theoretical guarantees and convergence results are stated for this case. We show that the network weights can be computed by solving an explicit equation at the reduced MMV dimensions which also admits a closed-form solution. Toward more practical problems, we then considered convolutional observation models and show that the proposed architecture and the analytical weight computation can be further simplified and thus open new directions for convolutional neural networks. Finally, we evaluate the unfolded algorithms in numerical experiments and discuss connections to other sparse recovering algorithms.

Discretized tensor-based model of total focusing method: A sparse regularization approach for enhanced ultrasonic phased array imaging

The total focusing method (TFM) is considered as the standard in ultrasonic phased array imaging and plays a vital role in industrial non-destructive testing (NDT). By utilizing the full matrix capture (FMC) dataset, the TFM can focus on every point within the specified imaging region, and is more accurate than the traditional ultrasonic phased array imaging methods. However, the TFM is essentially a delay and sum technique that often operates linearly on the time-domain signals and takes no prior information into account, so its image quality remains inadequate when dealing with defects in close proximity or scattering materials. To address this problem, this paper formulates the imaging principle of the TFM as a Boolean matrix and establishes the discretized tensor-based model. Subsequently, the model is addressed by employing the sparse regularization strategy, taking into some characteristics of industrial NDT. Regarding the solution algorithm, due to the generation of negative values by the fast iterative shrinkage threshold algorithm (FISTA), this paper introduces the rectified linear unit (ReLU) function as a non-negative constraint and presents a dedicated solution algorithm (ReLU-FISTA) for acquiring detection results. Through verification of simulation and experiment, the proposed approach exhibits superior capabilities of defect characterization and noise suppression when compared to the TFM, leading to substantial enhancements in image quality.

Terahertz wave imaging using time domain iterative shrink threshold method

We propose a new linear imaging method that combines the iterative shrinkage thresholding algorithm (ISTA) with the synthetic aperture (SA) method to achieve high resolution. Unlike other ISTA methods that use a single frequency component of signals, we use time domain signals directly as measurement, which allows for simultaneous wide bandwidth calculation and potentially better imaging results. Additionally, we simplify the backward estimation process by using impulse back-projection based on the SA method, which reduces the computing cost compared to the ISTA method. Numerical simulation and experimental measurement results show that the image resolution is similar to that of our previous non-linear imaging method, and two reflectors with different reflectivity are imaged linearly. These comparison results demonstrate the effectiveness of our proposed method in achieving high-resolution and maintaining the linearity of target reflectivity.

Application of Sparse Regularization in Spherical Radial Basis Functions-Based Regional Geoid Modeling in Colorado

Spherical radial basis function (SRBF) is an effective method for calculating regional gravity field models. Calculating gravity field models with high accuracy and resolution requires dense basis functions, resulting in complex models. This study investigated the application of sparse regularization in SRBFs-based regional gravity field modeling. L1-norm regularization, also known as the least absolute shrinkage selection operator (LASSO), was employed in the parameter estimation procedure. LASSO differs from L2-norm regularization in that the solution obtained by LASSO is sparse, specifically with a portion of the parameters being zero. A sparse model would be advantageous for improving the numerical efficiency by reducing the number of SRBFs. The optimization problem of the LASSO was solved using the fast iterative shrinkage threshold algorithm, which is known for its high efficiency. The regularization parameter was selected using the Akaike information criterion. It was specifically tailored to the L1-norm regularization problem. An approximate covariance matrix of the estimated parameters in the sparse solution was analytically constructed from a Bayesian viewpoint. Based on the remove–compute–restore technique, a regional geoid model of Colorado (USA) was calculated. The numerical results suggest that the LASSO adopted in this study provided competitive results compared to Tikhonov regularization; however, the number of basis functions in the final model was less than 25% of the Tikhonov regularization. Without significantly reducing model accuracy, the LASSO solution provides a very simple model. This is the first study to apply the LASSO to SRBFs-based modeling of the regional gravity field in real gravity observation data.

Frequency analysis of passive sonar signals using multiple-measurement physics-informed neural network

We propose a deep learning based frequency analysis for passive sonar signals by considering a linear system based on a relation between time and frequency domain signal representations. An adaptive learned iterative shrinkage thresholding algorithm (Ada-LISTA) is utilized as the deep learning architecture to solve the linear system effectively and to ensure generalization performance. To obtain more reliable solutions, we involve the linear system in loss function during training. Futhermore, in order to reduce noise effectively, we employ multiple measurements from time and space domains, which have common frequency components of passive sonar signals. Thus, the loss function is modified for solutions from the multiple measurements to have shared frequency components. We conduct experiments using synthetic and underwater in-situ data to examine performance of frequency analysis considering the linear system in designing the architecture and finding the optimal weights. The frequency analysis demonstrates superior performance in detecting frequency components and reducing noise, compared to fast Fourier transform and sparse Bayesian learning with low computational burden during its application to the test measurements.

TEFISTA-Net: A learnable method for high-resolution range profile reconstruction with low-frequency ultra-wideband radar

In low-frequency ultra-wideband (LFW) radar, the reconstruction of high-resolution range profile (HRRP) is an important task. The state-of-the-art compressive sensing (CS) method for HRRP reconstruction based on geometrical theory of diffraction (GTD) requires manual algorithm parameter tuning and is computationally expensive. In this paper, we propose a deep learning-based CS method for LFW radar HRRP reconstruction. We design a neural network architecture, i.e., target enhancement-based FISTA-Net (TEFISTA-Net), by unrolling the fast iterative shrinkage thresholding algorithm (FISTA). A new loss function based on the target-to-background ratio (TBR) is introduced for network training with the target enhancement capability in low signal-to-noise ratio (SNR) scenarios. The algorithm parameters in the traditional CS methods are substituted by network parameters learned from training data, getting rid of manual parameter tuning. In addition, simple convolution operations in our new method lead to lower computational complexity compared with existing methods. Experimental results based on diverse data show that the proposed method leads to higher computational efficiency with similar or better performance in comparison with existing state-of-the-art methods.

Nonsparse SAR Scene Imaging Network Based on Sparse Representation and Approximate Observations

Sparse-representation-based synthetic aperture radar (SAR) imaging technology has shown superior potential in the reconstruction of nonsparse scenes. However, many existing compressed sensing (CS) methods with sparse representation cannot obtain an optimal sparse basis and only apply to the sensing matrix obtained by exact observation, resulting in a low image quality occupying more storage space. To reduce the computational cost and improve the imaging performance of nonsparse scenes, we formulate a deep learning SAR imaging method based on sparse representation and approximated observation deduced from the chirp-scaling algorithm (CSA). First, we incorporate the CSA-derived approximated observation model and a nonlinear transform function within a sparse reconstruction framework. Second, an iterative shrinkage threshold algorithm is adopted to solve this framework, and the solving process is unfolded as a deep SAR imaging network. Third, a dual-path convolutional neural network (CNN) block is designed in the network to achieve the nonlinear transform, dramatically improving the sparse representation capability over conventional transform-domain-based CS methods. Last, we improve the CNN block to develop an enhanced version of the deep SAR imaging network, in which all the parameters are layer-varied and trained by supervised learning. The experiments demonstrate that our proposed two imaging networks outperform conventional CS-driven and deep-learning-based methods in terms of computing efficiency and reconstruction performance of nonsparse scenes.

Automatic Modulation Recognition Based on a New Deep K-SVD Denoising Algorithm

Automatic modulation recognition has a wide range of applications in wireless communication. To solve the problem that the previous methods convert signal modulation recognition into image recognition, leading to increased time costs and information loss, an automatic modulation recognition approach consisting of the improved deep K-SVD denoising algorithm is suggested. First, the effectiveness of the model for random sine wave denoising is demonstrated by simulation. Second, the original I/Q signals are fed into the modified deep K-SVD model for denoising, skipping the complicated image processing steps. Finally, the noise-reduced signals are input into a multi-channel convolutional long short-term neural network to complete the modulation recognition. To solve the slow convergence problem of Iterative Shrinkage Thresholding Algorithms in sparse coding, the Fast Iterative Shrinkage Thresholding Algorithm is adopted to improve the computational efficiency and obtain a better denoising effect. The experiments show that the improved model has an average recognition accuracy of 91.26% at different SNRs from -2 dB to 18 dB, which is better than the state-of-the-art modulation recognition models.

Broadband Acoustic Modal Identification by Array Nonsynchronous Measurements with One Reference

This paper proposes a broadband acoustic modal identification method based on nonsynchronous measurements from a microphone array, which overcomes the acoustic modal aliasing problem caused by a limited number of microphones. In the proposed method, the number of measurement points is augmented by sequentially rotating a circular microphone array that is circumferentially mounted in the wall of the compressor duct. This technique employs only one reference microphone to enhance the robustness of the cross-spectral matrix completion for nonsynchronous measurements. The fast iterative shrinkage threshold algorithm is adopted to iteratively complete the cross-spectral matrix of all measurement points for acoustic modal identification. The in-duct acoustic modal coefficients of the broadband noise are then determined using the least-squares method with the sound pressure vector recovered from the completed cross-spectral matrix. The accuracy of the cross-spectral matrix and the identified acoustic modals is verified through numerical simulations. The proposed method is validated by experimental results from an axial flow compressor. The results show that it can effectively suppress the modal aliasing problem and identify the azimuthal modes of compressor broadband noise.