Iterative Thresholding Algorithm Research Articles

[formula omitted] sparsity regularization for nonlinear ill-posed problems

In this paper, the α‖⋅‖ℓ1−β‖⋅‖ℓ2 sparsity regularization with parameters α≥β≥0 is studied for nonlinear ill-posed inverse problems. The well-posedness of the regularization is investigated. Compared to the case where α>β≥0, the results for the case α=β>0 are weaker due to the lack of coercivity and Radon-Riesz property of the regularization term. Under certain conditions on the nonlinearity of F, sparsity is shown for every minimizer of the α‖⋅‖ℓ1−β‖⋅‖ℓ2 regularized inverse problem. Moreover, for the case α>β≥0, convergence rates O(δ12) and O(δ) are proved for the regularized solution towards a sparse exact solution, under different yet commonly adopted conditions on the nonlinearity of F. The iterative soft thresholding algorithm is shown to be useful to solve the α‖⋅‖ℓ1−β‖⋅‖ℓ2 regularized problem for nonlinear ill-posed equations. Numerical results illustrate the efficiency of the proposed method.

Deep learning network with new weighting strategy for ISAR image enhancement

ABSTRACT In inverse synthetic aperture radar (ISAR) imaging, the conventional range-Doppler (RD) algorithm cannot obtain satisfactory imaging results for sparse aperture. Compressed sensing (CS) imaging methods, such as the typical iterative shrinkage and thresholding algorithm (ISTA), are often used in sparse aperture radar imaging. However, CS-based methods have high computational complexity and difficulty in setting parameters. To overcome the shortcomings, a new deep unfolding network named complex-valued weighted learning ISTA (CV-WLISTA) is proposed in this paper. The new network is very efficient and can learn the parameters adaptively. Based on ISTA and a new weighting strategy, it takes advantage of both the sparsity and the structural property of ISAR images to improve imaging performance. The experimental results show that CV-WLISTA has better reconstruction performance and higher efficiency compared with the traditional algorithms. Therefore, CV-WLISTA is an efficient ISAR imaging method with excellent performance.

An Adaptive Multi-D-Norm-Driven Sparse Unfolding Deconvolutional Network for Bearing Fault Diagnosis.

Impulsive blind deconvolution (IBD) is a popular method to recover impulsive sources for bearing fault diagnosis. Its underpinnings are in the design of objective functions based on prior knowledge of impulsive sources and a transfer function to describe transmission path influences. However, popular objective functions cannot retain waveform impulsiveness and periodicity cyclostationarity simultaneously, and the single convolution operation of IBD methods is insufficient to describe transmission paths composed of multiple linear and nonlinear units. Inspired by the MaxPooling period modulation intensity (MPMI) and convolutional sparse learning (CSL), an adaptive multi-D-norm-driven sparse unfolding deconvolution network (AMD-SUDN) is proposed in this paper. The core strategy is that one target vector with simultaneous impulsiveness and cyclostationarity is constructed automatically through the MPMI; then, this vector is substituted into the multi D-norm to design objective functions. Moreover, an iterative soft threshold algorithm (ISTA) for the CSL model is derived, and its iterative steps are unfolded into one deconvolution network. The algorithm's performance and the hyperparameter configuration are investigated by a set of numerical simulations. Finally, the proposed AMD-SUDN is applied to detect the impulsive features of bearing faults. All comparative results verify that the proposed AMD-SUDN achieves a better deconvolution accuracy than state-of-the-art IBD methods.

A Designed Thresholding Operator for Low-Rank Matrix Completion

In this paper, a new thresholding operator, namely, designed thresholding operator, is designed to recover the low-rank matrices. With the change of parameter in designed thresholding operator, the designed thresholding operator can apply less bias to the larger singular values of a matrix compared with the classical soft thresholding operator. Based on the designed thresholding operator, an iterative thresholding algorithm for recovering the low-rank matrices is proposed. Numerical experiments on some image inpainting problems show that the proposed thresholding algorithm performs effectively in recovering the low-rank matrices.

Migration through Resolution Cell Correction and Sparse Aperture ISAR Imaging for Maneuvering Target Based on Whale Optimization Algorithm-Fast Iterative Shrinkage Thresholding Algorithm.

Targets faced by inverse synthetic aperture radar (ISAR) are often non-cooperative, with target maneuvering being the main manifestation of this non-cooperation. Maneuvers cause ISAR imaging results to be severely defocused, which can create huge difficulties in target identification. In addition, as the ISAR bandwidth continues to increase, the impact of migration through resolution cells (MTRC) on imaging results becomes more significant. Target non-cooperation may also result in sparse aperture, leading to the failure of traditional ISAR imaging algorithms. Therefore, this paper proposes an algorithm to realize MTRC correction and sparse aperture ISAR imaging for maneuvering targets simultaneously named whale optimization algorithm-fast iterative shrinkage thresholding algorithm (WOA-FISTA). In this algorithm, FISTA is used to perform MTRC correction and sparse aperture ISAR imaging efficiently and WOA is adopted to estimate the rotational parameter to eliminate the effects of maneuvering on imaging results. Experimental results based on simulation and measured datasets prove that the proposed algorithm implements sparse aperture ISAR imaging and MTRC correction for maneuvering targets simultaneously. The proposed algorithm achieves better results than traditional algorithms under different signal-to-noise ratio conditions.

EIDNet: Extragradient-based iterative denoising network for image compressive sensing reconstruction

Compressive sensing (CS) is a kind of hardware-friendly technology in effective image reconstruction. Most of the existing CS reconstruction methods can be classified into model-driven iterative optimization methods and data-driven neural network methods. These two types of methods have their own problems with computational complexity and theoretical interpretability. Motivated by the deep unfolding theory, we develop the model-driven Extragradient-based Iterative Denoising Network, dubbed EIDNet, which is based on the denoising model of the iterative shrinkage threshold algorithm (ISTA). By unfolding the iterative denoising model into the deep learning network, EIDNet addresses the interpretability problem of neural network methods in CS reconstruction. Each denoising iteration of the optimization algorithm is mapped into one layer of the reconstruction network in EIDNet. The extragradient method is adopted to accelerate network convergence. In addition, we introduce a learning-based sampling and initialization network for adaptive sampling and initialization. Extensive experiments show that the proposed EIDNet can obtain better image CS reconstruction performance than other state-of-the-art CS methods while maintaining satisfactory reconstruction efficiency.

Leveraging local neighborhood features in 3D unstructured point cloud data for geometry-based automated damage delineation

In a post-earthquake environment, manual inspection is resource-intensive and subjective. This article presents a real-world damage assessment study of the Improved Iterative Bi-Level Thresholding (IIBLT) algorithm. Damaged regions are delineated using IIBLT based on the local point curvature. The methodology is tested using a unique dataset: Mono Temporal Disaster City 3D Dataset (C3DO). C3DO is a point cloud data generated by a 3D reconstruction of a controlled post-earthquake environment, which presents challenges for concrete façade damage assessment algorithms. Results from IIBLT demonstrated a 16% increase in Matthews Correlation Coefficient (MCC) and Intersection over Union (mGIoU), compared to the current state-of-the-art methods. This corresponds to improving accuracy and completeness in defect region identification without the dependence on knowledge of damaged regions. Results from this study highlight the novel contributions of the IIBLT algorithm including the elimination of parameter tuning, and demonstrate the applicability of the novel iterative framework for concrete surface defect delineation at large.

Robust treatment planning for small animal radio-neuromodulation using focused kV x-ray beams.

In preclinical radio-neuromodulation research, small animal experiments are pivotal for unraveling radiobiological mechanism, investigating prescription and planning techniques, and assessing treatment effects and toxicities. However, the target size inside a rat brain is typically in the order of sub-millimeters. The small target inside the visual cortex neural region in rat brain with a diameter of around 1mm was focused in this work to observe the physiological change of this region. Delivering uniform doses to the small target while sparing health tissues is challenging. Focused kV x-ray technique based on modern x-ray polycapillary focusing lens is a promising modality for small animal radio-neuromodulation. The current manual planning method could lead to sub-optimal plans, and the positioning uncertainties due to mechanical accuracy limitations, animal immobilization, and robotic arm motion are not considered. This work aims to design a robust inverse planning method to optimize the intensities of focused kV x-ray beams located in beam trajectories to irradiate small mm-sized targets in rat brains for radio-neuromodulation. Focused kV x-ray beams were generated through polycapillary x-ray focusing lenses on achieving small (≤0.3mm) focus perpendicular to the beam. The beam trajectories were manually designed in 3D space in scanning-while-rotating mode. Geant4 Monte Carlo (MC) simulation generated a dose calculation matrix for each focused kV x-ray beam located in beam trajectories. In the proposed robust inverse planning method, an objective function combining a voxel-wise stochastic programming approach and L1 norm regularization was established to overcome the positioning uncertainties and obtain a high-quality plan. The fast iterative shrinkage thresholding algorithm (FISTA) was utilized to solve the objective function and obtain the optimal intensities. Four cases were employed to validate the feasibility and effectiveness of the proposed method. The manual and non-robust inverse planning methods were also implemented for comparison. The proposed robust inverse planning method achieved superior dose homogeneity and higher robustness against positioning uncertainties. On average, the clinical target volume (CTV) homogeneity index (HI) of robust inverse plan improved to 13.3 from 22.9 in non-robust inverse plan and 53.8 in manual plan if positioning uncertainties were also present. The average bandwidth at D90 was reduced by 6.5Gy in the robust inverse plan, compared to 9.6Gy in non-robust inverse plan and 12.5Gy in manual plan. The average bandwidth at D80 was reduced by 3.4Gy in robust inverse plan, compared to 5.5Gy in non-robust inverse plan and 8.5Gy in manual plan. Moreover, the dose delivery time of manual plan was reduced by an average reduction of 54.7% with robust inverse plan and 29.0% with non-robust inverse plan. Compared to manual and non-robust inverse planning methods, the robust inverse planning method improved the dose homogeneity and delivery efficiency and was resistant to the uncertainties, which are crucial for radio-neuromodulation utilizing focused kV x-rays.

On convergence of iterative thresholding algorithms to approximate sparse solution for composite nonconvex optimization

AbstractThis paper aims to find an approximate true sparse solution of an underdetermined linear system. For this purpose, we propose two types of iterative thresholding algorithms with the continuation technique and the truncation technique respectively. We introduce a notion of limited shrinkage thresholding operator and apply it, together with the restricted isometry property, to show that the proposed algorithms converge to an approximate true sparse solution within a tolerance relevant to the noise level and the limited shrinkage magnitude. Applying the obtained results to nonconvex regularization problems with SCAD, MCP and $$\ell _p$$ ℓ p penalty ($$0\le p \le 1$$ 0 ≤ p ≤ 1 ) and utilizing the recovery bound theory, we establish the convergence of their proximal gradient algorithms to an approximate global solution of nonconvex regularization problems. The established results include the existing convergence theory for $$\ell _1$$ ℓ 1 or $$\ell _0$$ ℓ 0 regularization problems for finding a true sparse solution as special cases. Preliminary numerical results show that our proposed algorithms can find approximate true sparse solutions that are much better than stationary solutions that are found by using the standard proximal gradient algorithm.

A Fast Intersection of Confidence Intervals Method-Based Adaptive Thresholding for Sparse Image Reconstruction Using the Matrix Form of the Wavelet Transform

One of the frequently used classes of sparse reconstruction algorithms is based on the iterative shrinkage/thresholding procedure, in which the thresholding parameter controls a trade-off between the algorithm’s accuracy and execution time. In order to avoid this trade-off, we propose using a fast intersection of confidence intervals method in order to adaptively control the threshold value throughout the iterations of the reconstruction algorithm. We have upgraded the two-step iterative shrinkage thresholding algorithm with a such procedure, improving its performance. The proposed algorithm, denoted as the FICI-TwIST, along with a few selected state-of-the-art sparse reconstruction algorithms, has been tested on the classical problem of image recovery by emphasizing the image sparsity in the discrete cosine and the discrete wavelet domain. Furthermore, we have derived a single wavelet transformation matrix which avoids wrapping effects, thereby achieving significantly faster execution times as compared to a more traditional function-based transformation. The obtained results indicate the competitive performance of the proposed algorithm, even in cases where all algorithm parameters have been individually fine-tuned for best performance.

MsDC-DEQ-Net: Deep Equilibrium Model (DEQ) with Multiscale Dilated Convolution for Image Compressive Sensing (CS)

Compressive sensing (CS) is a technique that enables the recovery of sparse signals using fewer measurements than traditional sampling methods. To address the computational challenges of CS reconstruction, our objective is to develop an interpretable and concise neural network model for reconstructing natural images using CS. We achieve this by mapping one step of the iterative shrinkage thresholding algorithm (ISTA) to a deep network block, representing one iteration of ISTA. To enhance learning ability and incorporate structural diversity, we integrate aggregated residual transformations (ResNeXt) and squeeze-and-excitation mechanisms into the ISTA block. This block serves as a deep equilibrium layer connected to a semi-tensor product network for convenient sampling and providing an initial reconstruction. The resulting model, called MsDC-DEQ-Net, exhibits competitive performance compared to state-of-the-art network-based methods. It significantly reduces storage requirements compared to deep unrolling methods, using only one iteration block instead of multiple iterations. Unlike deep unrolling models, MsDC-DEQ-Net can be iteratively used, gradually improving reconstruction accuracy while considering computation tradeoffs. Additionally, the model benefits from multiscale dilated convolutions, further enhancing performance.

PISTA: Preconditioned Iterative Soft Thresholding Algorithm for Graphical Lasso

pISTA: Preconditioned Iterative Soft Thresholding Algorithm for Graphical Lasso

Self-Supervised Medical Image Denoising Based on WISTA-Net for Human Healthcare in Metaverse.

Medical image processing plays an important role in the interaction of real world and metaverse for healthcare. Self-supervised denoising based on sparse coding methods, without any prerequisite on large-scale training samples, has been attracting extensive attention for medical image processing. Whereas, existing self-supervised methods suffer from poor performance and low efficiency. In this paper, to achieve state-of-the-art denoising performance on the one hand, we present a self-supervised sparse coding method, named the weighted iterative shrinkage thresholding algorithm (WISTA). It does not rely on noisy-clean ground-truth image pairs to learn from only a single noisy image. On the other hand, to further improve denoising efficiency, we unfold the WISTA to construct a deep neural network (DNN) structured WISTA, named WISTA-Net. Specifically, in WISTA, motivated by the merit of the lp-norm, WISTA-Net has better denoising performance than the classical orthogonal matching pursuit (OMP) algorithm and the ISTA. Moreover, leveraging the high-efficiency of DNN structure in parameter updating, WISTA-Net outperforms the compared methods in denoising efficiency. In detail, for a 256 by 256 noisy image, the running time of WISTA-Net is 4.72 s on the CPU, which is much faster than WISTA, OMP, and ISTA by 32.88 s, 13.06 s, and 6.17 s, respectively.

Iterative singular tube hard thresholding algorithms for tensor recovery

Iterative singular tube hard thresholding algorithms for tensor recovery

Algorithm unfolding for block-sparse and MMV problems with reduced training overhead

In this study, we consider algorithm unfolding for the multiple measurement vector (MMV) problem in the case where only few training samples are available. Algorithm unfolding has been shown to empirically speed-up in a data-driven way the convergence of various classical iterative algorithms, but for supervised learning, it is important to achieve this with minimal training data. For this, we consider learned block iterative shrinkage thresholding algorithm (LBISTA) under different training strategies. To approach almost data-free optimization at minimal training overhead, the number of trainable parameters for algorithm unfolding has to be substantially reduced. We therefore explicitly propose a reduced-size network architecture based on the Kronecker structure imposed by the MMV observation model and present the corresponding theory in this context. To ensure proper generalization, we then extend the analytic weight approach by Liu and Chen to LBISTA and the MMV setting. Rigorous theoretical guarantees and convergence results are stated for this case. We show that the network weights can be computed by solving an explicit equation at the reduced MMV dimensions which also admits a closed-form solution. Toward more practical problems, we then considered convolutional observation models and show that the proposed architecture and the analytical weight computation can be further simplified and thus open new directions for convolutional neural networks. Finally, we evaluate the unfolded algorithms in numerical experiments and discuss connections to other sparse recovering algorithms.

Gravity inversion using L0 norm for sparse constraints

SUMMARY Gravity surveys constitute an important method for investigating the Earth's interior based on density contrasts related to Earth material differentials. Because lithology depends on the environment and the period of formation, there are generally clear boundaries between rocks with different lithologies. Inversions with convex functions for approximating the L0 norm are used to detect boundaries in reconstructed models. Optimizations can easily be found because of the convex transformations; however, the volume of the reconstructed model depends on the weighting parameter and the density constraint rather than the model sparsity. To determine and adapt the modelling size, a novel non-convex framework for gravity inversion is proposed. The proposed optimization aims to directly reduce the L0 norm of the density matrix. An improved iterative hard thresholding algorithm is developed to linearly reduce the L0 penalty during the inner iteration. Accordingly, it is possible to determine the modelling scale during the iteration and achieve an expected scale for the reconstructed model. Both simple and complex model experiments demonstrate that the proposed method efficiently reconstructs models. In addition, granites formed during the Yanshanian and Indosinian periods in the Nanling region, China, are reconstructed according to the modelling size evaluated in agreement with the magnetotelluric profile and density statistics of rock samples. The known ores occur at the contact zones between the sedimentary rocks and the reconstructed Yanshanian granites. The ore-forming bodies, periods, and processes are identified, providing guidance for further deep resource exploration in the study area.

Discretized tensor-based model of total focusing method: A sparse regularization approach for enhanced ultrasonic phased array imaging

The total focusing method (TFM) is considered as the standard in ultrasonic phased array imaging and plays a vital role in industrial non-destructive testing (NDT). By utilizing the full matrix capture (FMC) dataset, the TFM can focus on every point within the specified imaging region, and is more accurate than the traditional ultrasonic phased array imaging methods. However, the TFM is essentially a delay and sum technique that often operates linearly on the time-domain signals and takes no prior information into account, so its image quality remains inadequate when dealing with defects in close proximity or scattering materials. To address this problem, this paper formulates the imaging principle of the TFM as a Boolean matrix and establishes the discretized tensor-based model. Subsequently, the model is addressed by employing the sparse regularization strategy, taking into some characteristics of industrial NDT. Regarding the solution algorithm, due to the generation of negative values by the fast iterative shrinkage threshold algorithm (FISTA), this paper introduces the rectified linear unit (ReLU) function as a non-negative constraint and presents a dedicated solution algorithm (ReLU-FISTA) for acquiring detection results. Through verification of simulation and experiment, the proposed approach exhibits superior capabilities of defect characterization and noise suppression when compared to the TFM, leading to substantial enhancements in image quality.

Terahertz wave imaging using time domain iterative shrink threshold method

We propose a new linear imaging method that combines the iterative shrinkage thresholding algorithm (ISTA) with the synthetic aperture (SA) method to achieve high resolution. Unlike other ISTA methods that use a single frequency component of signals, we use time domain signals directly as measurement, which allows for simultaneous wide bandwidth calculation and potentially better imaging results. Additionally, we simplify the backward estimation process by using impulse back-projection based on the SA method, which reduces the computing cost compared to the ISTA method. Numerical simulation and experimental measurement results show that the image resolution is similar to that of our previous non-linear imaging method, and two reflectors with different reflectivity are imaged linearly. These comparison results demonstrate the effectiveness of our proposed method in achieving high-resolution and maintaining the linearity of target reflectivity.

Application of Sparse Regularization in Spherical Radial Basis Functions-Based Regional Geoid Modeling in Colorado

Spherical radial basis function (SRBF) is an effective method for calculating regional gravity field models. Calculating gravity field models with high accuracy and resolution requires dense basis functions, resulting in complex models. This study investigated the application of sparse regularization in SRBFs-based regional gravity field modeling. L1-norm regularization, also known as the least absolute shrinkage selection operator (LASSO), was employed in the parameter estimation procedure. LASSO differs from L2-norm regularization in that the solution obtained by LASSO is sparse, specifically with a portion of the parameters being zero. A sparse model would be advantageous for improving the numerical efficiency by reducing the number of SRBFs. The optimization problem of the LASSO was solved using the fast iterative shrinkage threshold algorithm, which is known for its high efficiency. The regularization parameter was selected using the Akaike information criterion. It was specifically tailored to the L1-norm regularization problem. An approximate covariance matrix of the estimated parameters in the sparse solution was analytically constructed from a Bayesian viewpoint. Based on the remove–compute–restore technique, a regional geoid model of Colorado (USA) was calculated. The numerical results suggest that the LASSO adopted in this study provided competitive results compared to Tikhonov regularization; however, the number of basis functions in the final model was less than 25% of the Tikhonov regularization. Without significantly reducing model accuracy, the LASSO solution provides a very simple model. This is the first study to apply the LASSO to SRBFs-based modeling of the regional gravity field in real gravity observation data.

Frequency analysis of passive sonar signals using multiple-measurement physics-informed neural network

We propose a deep learning based frequency analysis for passive sonar signals by considering a linear system based on a relation between time and frequency domain signal representations. An adaptive learned iterative shrinkage thresholding algorithm (Ada-LISTA) is utilized as the deep learning architecture to solve the linear system effectively and to ensure generalization performance. To obtain more reliable solutions, we involve the linear system in loss function during training. Futhermore, in order to reduce noise effectively, we employ multiple measurements from time and space domains, which have common frequency components of passive sonar signals. Thus, the loss function is modified for solutions from the multiple measurements to have shared frequency components. We conduct experiments using synthetic and underwater in-situ data to examine performance of frequency analysis considering the linear system in designing the architecture and finding the optimal weights. The frequency analysis demonstrates superior performance in detecting frequency components and reducing noise, compared to fast Fourier transform and sparse Bayesian learning with low computational burden during its application to the test measurements.