Iterative Recovery Research Articles

SA-UCBSS: Sparsity-Based Adaptive Underdetermined Convolutive Blind Source Separation

Blind source separation is the process of separating unknown source signals based only on the received mixing signals when the source signal cannot be accurately known. Traditional methods are mainly based on linear approximate transformation models that often have large systematic errors in high reverberation environments, resulting in poor separation performance. To avoid these limitations, a sparsity-based adaptive underdetermined convolutive blind source separation (SA-UCBSS) algorithm is proposed by unifying a convolutive approximate transformation model and an Lp(0<p≤1)-norm regularisation penalty. A crosstalk canceller is designed for room impulse response pretreatment with p-norm optimisation to eliminate audible echoes, and a sparse optimisation scheme is designed for the parameter initialisation to accelerate the convergence of the iterative recovery process. Subsequently, a convolutive approximate transformation model is built to achieve a more accurate estimation of the mixing system. The SA-UCBSS algorithm is proposed to achieve better source separation by adaptively tuning the value of parameter p. Experimental results including BSS of the synthesised and live-recorded speech convolutive mixtures showed that the SA-UCBSS algorithm had superior separating properties and better robustness than several popular algorithms.

Unified interpretation of MAOS responses via experimentally decomposed material functions

Oscillatory shear testing, used to characterize the viscoelastic response of soft materials, is often divided into small, medium, and large amplitude oscillatory shear (SAOS, MAOS, and LAOS) regions. SAOS is a common test that gives us a unified analysis and interpretation of linear viscoelastic behavior, whereas understanding MAOS and LAOS is still an active area of research. While numerous mathematical techniques have been proposed, a consensus interpretation is still missing. Recently, our understanding of nonlinear behavior in the LAOS regime has been developed using iterative recovery tests. Recovery rheology decomposes the strain into two components, allowing an unambiguous interpretation of the nonlinear behavior in terms of sequences of recoverable and unrecoverable processes. In this work, we revisit the MAOS material functions for polyvinyl alcohol-borax hydrogel and worm-like micelles using recovery rheology. We show that two mathematical formalisms, the Chebyshev and sequence of physical processes analyses, provide competing physical interpretations when they are derived from the total strain, but provide unified interpretations when describing the decomposed strains. We, therefore, show that what has often been treated as a mathematical problem can instead be solved experimentally by acknowledging the extra information provided by recovery rheology.

Generalization error bounds for iterative recovery algorithms unfolded as neural networks

Abstract Motivated by the learned iterative soft thresholding algorithm (LISTA), we introduce a general class of neural networks suitable for sparse reconstruction from few linear measurements. By allowing a wide range of degrees of weight-sharing between the flayers, we enable a unified analysis for very different neural network types, ranging from recurrent ones to networks more similar to standard feedforward neural networks. Based on training samples, via empirical risk minimization, we aim at learning the optimal network parameters and thereby the optimal network that reconstructs signals from their low-dimensional linear measurements. We derive generalization bounds by analyzing the Rademacher complexity of hypothesis classes consisting of such deep networks, that also take into account the thresholding parameters. We obtain estimates of the sample complexity that essentially depend only linearly on the number of parameters and on the depth. We apply our main result to obtain specific generalization bounds for several practical examples, including different algorithms for (implicit) dictionary learning, and convolutional neural networks.

EiCSNet: Efficient Iterative Neural Network for Compressed Sensing Reconstruction

The rapid growth of sensing data demands compressed sensing (CS) in order to achieve high-density storage and fast data transmission. Deep neural networks (DNNs) have been under intensive development for the reconstruction of high-quality images from compressed data. However, the complicated auxiliary structures of DNN models in pursuit of better recovery performance lead to low computational efficiency and long reconstruction times. Furthermore, it is difficult for conventional neural network designs to reconstruct extra-high-frequency information at a very low sampling rate. In this work, we propose an efficient iterative neural network for CS reconstruction (EiCSNet). An efficient gradient extraction module is designed to replace the complex auxiliary structures in order to train the DNNs more efficiently. An iterative enhancement network is applied to make full use of the limited information available in CS for better iterative recovery. In addition, a frequency-aware weighted loss is further proposed for better image restoration quality. Our proposed compact model, EiCSNet2*1, improved the performance slightly and was nearly seven times faster than its counterparts, which shows that it has a highly efficient network design. Additionally, our complete model, EiCSNet6*1, achieved the best effect at this stage, where the average PSNR was improved by 0.37 dB for all testing sets and sampling rates.

Seasonal Cycles of High Mountain Asia Glacier Surface Elevation Detected by ICESat‐2

AbstractHigh Mountain Asia (HMA) glaciers are a key component of mountain hydrology and their seasonal ablation contributes significantly to the annual runoff of several large river systems that protect large populations from water stress. However, the seasonal cycle of glacial change remains unclear. Recent NASA's Ice, Cloud, and Land Elevation Satellite (ICESat‐2) provides an opportunity to survey changes in the glacier surface elevation in unprecedented detail. Here, we used a new approach—which is based on a developed elevation bin cluster with a sliding filter and an iterative recovery method—to obtain the complete seasonal cycle of glacier thickness changes in HMA. We generally identified three types of glaciers: (a) glaciers in the westerly dominant area (WDA) (b) glaciers in the Indian monsoon area (IMA), and (c) glaciers in the transition area (TA). The peak thickness values for the IMA, WDA, and TA glaciers occur in February, May, and June, respectively. Most glaciers in WDA and IMA show a strong seasonal cycle with variations of more than 2 m, while the TA glaciers are mainly characterized by weak changes of up to 1 m. Overall, our results show that the glaciers can provide 67–138 Gt of meltwater per year to downstream. These results provide crucial information for the calibration of models that project glacier response to climate change and simulate the downstream water availability. Our methodology facilitates the application of ICESat‐2 to seasonal tracking of glacier variability in global alpine areas.

Transferring Deep Gaussian Denoiser for Compressed Sensing MRI Reconstruction

Deep neural networks have achieved the most outstanding performance in compressed sensing magnetic resonance imaging (CS-MRI) reconstruction by learning the potential structures of images from a large number of training samples. However, the required data comprising hundreds of subjects are usually rare. In this article, we remedy this problem by transferring the easy-to-get deep Gaussian denoisers trained with natural images for artifact reduction in the iterative recovery process without the use of full-sampled MRI data. To this end, we first train a set of deep Gaussian networks with natural images and then incorporate them into a play-and-plug framework that is built by modifying the proximal gradient algorithm with the classical momentum strategy. Furthermore, a nonlocal denoiser is employed for efficiently removing the artifacts. It is found that the momentum strategy can make the statistical distribution of artifacts approximately be Gaussian, making the Gaussian denoisers available for CS-MRI. In the experiments, we verify the rationality of the established framework and show that our method consistently outperforms state-of-the-art methods.

Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

Countless signal processing applications include the reconstruction of signals from few indirect linear measurements. The design of effective measurement operators is typically constrained by the underlying hardware and physics, posing a challenging and often even discrete optimization task. While the potential of gradient-based learning via the unrolling of iterative recovery algorithms has been demonstrated, it has remained unclear how to leverage this technique when the set of admissible measurement operators is structured and discrete. We tackle this problem by combining unrolled optimization with Gumbel reparametrizations, which enable the computation of low-variance gradient estimates of categorical random variables. Our approach is formalized by GLODISMO (Gradient-based Learning of DIscrete Structured Measurement Operators). This novel method is easy-to-implement, computationally efficient, and extendable due to its compatibility with automatic differentiation. We empirically demonstrate the performance and flexibility of GLODISMO in several prototypical signal recovery applications, verifying that the learned measurement matrices outperform conventional designs based on randomization as well as discrete optimization baselines.

Compressive sensing of ECG signals using plug-and-play regularization

Compressive Sensing (CS) has recently attracted attention for ECG data compression. In CS, an ECG signal is projected onto a small set of random vectors. Recovering the original signal from such compressed measurements remains a challenging problem. Traditional recovery methods are based on solving a regularized minimization problem, where a sparsity-promoting prior is used. In this paper, we propose an alternative iterative recovery algorithm based on the Plug-and-Play (PnP) method, which has recently become popular for imaging problems. In PnP, a powerful denoiser is used to implicitly perform regularization, instead of using hand-crafted regularizers; this has been found to be more successful than traditional methods. In this work, we use a PnP version of the Proximal Gradient Descent (PGD) algorithm for ECG recovery. To ensure mathematical convergence of the PnP algorithm, the signal denoiser in question needs to satisfy some technical conditions. We use a high-quality ECG signal denoiser fulfilling this condition by learning a Bayesian prior for small-sized signal patches. This guarantees that the proposed algorithm converges to a fixed point irrespective of the initialization. Importantly, through extensive experiments, we show that the reconstruction quality of the proposed method is superior to that of state-of-the-art methods.

An Ensemble and Iterative Recovery Strategy Based kGNN Method to Edit Data with Label Noise

Learning label noise is gaining increasing attention from a variety of disciplines, particularly in supervised machine learning for classification tasks. The k nearest neighbors (kNN) classifier is often used as a natural way to edit the training sets due to its sensitivity to label noise. However, the kNN-based editor may remove too many instances if not designed to take care of the label noise. In addition, the one-sided nearest neighbor (NN) rule is unconvincing, as it just considers the nearest neighbors from the perspective of the query sample. In this paper, we propose an ensemble and iterative recovery strategy-based kGNN method (EIRS-kGNN) to edit data with label noise. EIRS-kGNN first uses the general nearest neighbors (GNN) to expand the one-sided NN rule to a binary-sided NN rule, taking the neighborhood of the queried samples into account. Then, it ensembles the prediction results of a finite set of ks in the kGNN to prudently judge the noise levels for each sample. Finally, two loops, i.e., the inner loop and the outer loop, are leveraged to iteratively detect label noise. A frequency indicator is derived from the iterative processes to guide the mixture approaches, including relabeling and removing, to deal with the detected label noise. The goal of EIRS-kGNN is to recover the distribution of the data set as if it were not corrupted. Experimental results on both synthetic data sets and UCI benchmarks, including binary data sets and multi-class data sets, demonstrate the effectiveness of the proposed EIRS-kGNN method.

Backscatter-Assisted Collision-Resilient LoRa Transmission.

Low-power wide-area networks (LPWANs), such as LoRaWAN, play an essential role and are expanding quickly in miscellaneous intelligent applications. However, the collision problem is also expanding significantly with the mass promotion of LPWAN nodes and providing collision-resilient techniques that are urgently needed for these applications. This paper proposes BackLoRa, a lightweight method that enables collision-resilient LoRa transmission with extra propagation information provided by backscatter tags. BackLoRa uses several backscatter tags to create multipath propagation features related to the LoRa nodes’ positions and offers a lightweight algorithm to extract the feature and correctly distinguish each LoRa node. Further, BackLoRa proposes a quick-phase acquisition algorithm with low time complexity that can carry out the iterative recovery of symbols for robust signal reconstructions in low-SNR conditions. Finally, comprehensive experiments were conducted in this study to evaluate the performance of BackLoRa systems. The experimental results show th compared with the existing scheme, our scheme can reduce the symbol error rate from 65.3% to 5.5% on average and improve throughput by 15× when SNR is −20 dB.

The nonlinear rheology of complex yield stress foods

Many foods have a yield stress that allows them to retain a desired shape at rest, but transition into a viscous fluid when being served or consumed. The determination of the yield stress of the food dictates how the foods are formed and packaged, how they are served, and how they are perceived when being eaten. Oscillatory shearing provides an ideal test protocol to map the rheology across a range of time and flow strength scales. We couple oscillatory shearing and an iterative recovery procedure to show that the yielding process is a continuous transition for two common yield stress foods. We show that unrecoverable processes from oscillatory tests are equivalent to the steady shear flow behavior. We show that this yielding behavior can be well approximated by a recently published model that treats yield stress materials as continuous viscoelastic fluids with a rate-dependent relaxation time and has parameters that can be obtained from the linear viscoelastic oscillatory frequency sweep and the steady shear flow curve.

LRSR-ADMM-Net: A Joint Low-Rank and Sparse Recovery Network for SAR Imaging

Synthetic aperture radar (SAR) imaging with sub-Nyquist sampled echo is a challenging task. Compressed sensing (CS) has been widely applied in this case to reconstruct the unambiguous image. The CS-based methods need to set the iterative parameters manually, but the appropriate parameters are usually difficult to obtain. Besides, such methods require a large number of iterations to obtain satisfactory results, which seriously restricts their practical applications. Moreover, the observation scene of SAR is not sparse in some cases. In this paper, we aim at proposing an efficient and effective imaging method for non-sparse observation scenes with reduced data. Firstly, considering the characteristics of non-sparse observation scenes in SAR imaging, we model the SAR imaging problem as a joint low-rank and sparse matrices recovery problem. After that, the iterative alternating direction method of multipliers (ADMM) to solve the above problem is unrolled into a layer-fixed deep neural network with trainable parameters, in which the learnable parameters are layer-varied. The threshold parameters, as well as the weight parameter between the sparse part and low-rank part of each layer, are learned adaptively instead of manually tuned. Experiments prove that the proposed LRSR-ADMM-Net is capable of reconstructing the non-sparse observed scene with high efficiency and precision. Particularly, the proposed LRSR-ADMM-Net yields better reconstruction performance while maintaining high computational efficiency compared with the state-of-the-art iterative recovery methods and the trainable sparse-based network methods.

A Novel Incremental Iterative Force Recovery Method for Second-Order Beam-Column Elements in Plastic Hinge Analysis of Steel Frames

This paper is concerned with an incremental iterative force recovery method in the second-order plastic hinge analysis of steel frames mainly modelled by a single element per member. Second-order beam-column elements are preferred in the direct analysis of steel frames due to their high accuracy and efficiency. However, formulations of these elements are complicated, and therefore they may have a problem of getting element force recovery in inelastic analysis. To overcome this difficulty, a novel incremental iterative force recovery method for second-order beam-column elements is proposed to perform plastic hinge analysis. The proposed method is derived more strictly and has good performance. Also, the section assemblage approach and the refined plastic hinge method are adopted in this study to consider the gradual degradation of section stiffness in the plastic hinge analysis. To verify the accuracy, efficiency and robustness of the proposed method, several benchmark examples are analyzed by the proposed method and compared with solutions reported by early researchers.

Multi-wavelet guided deep mean-shift prior for image restoration

Image restoration is essentially recognized as an ill-posed problem. A promising solution in recent years is incorporating deep network-driven priors into the iterative restoration procedure as constrained conditions. Among them, deep mean-shift prior utilizes the denoising autoencoder to play the role of prior updating. In this study, we present multiple wavelets guided deep mean-shift prior, which integrates the advantages of structural representation in the wavelet transform and learning ability in deep network. Specifically, by re-arranging the multi-view and multi-resolution features generated by multiple wavelet transforms as the input of denoising autoencoder, a more powerful prior information is learned. It benefits from the recurrent structure-preserving and multi-view complementary aggregation properties. We embed the learned prior information into the iterative recovery process and adopt proximal gradient descent to tackle it. Extensive experiments on image deblurring and compressed sensing tasks demonstrated significantly improved performances both visually and quantitatively.

A convergence analysis for iterative sparsification projection with soft-thresholding

The recently proposed iterative sparsification projection (ISP), a fast and robust sparse signal recovery algorithm framework, can be classified as smooth-ISP and nonsmooth-ISP. However, no convergence analysis has been established for the nonsmooth-ISP in the previous works. Motivated by this absence, the present paper provides a convergence analysis for ISP with soft-thresholding (ISP-soft) which is an instance of the nonsmooth-ISP. In our analysis, the composite operator of soft-thresholding and proximal projection is viewed as a fixed point mapping, whose nonexpansiveness plays a key role. Specifically, our convergence analysis for the sequence generated by ISP-soft can be summarized as follows: 1) For each inner loop, we prove that the sequence has a unique accumulation point which is a fixed point, and show that it is a Cauchy sequence; 2) for the last inner loop, we prove that the accumulation point of the sequence is a critical point of the objective function if the final value of the threshold satisfies a condition, and show that the corresponding objective values are monotonically nonincreasing. A numerical experiment is given to validate some of our results and intuitively present the convergence.

Attacking asymmetric cryptosystem based on phase truncated Fourier fransform by deep learning

Most of optical encryption systems are symmetric cryptosystems. The plaintext and the ciphertext in optical image encryption are related linearly. The security of the system needs to be strengthened. The asymmetric cryptosystem based on phase truncated Fourier transforms (PTFT) makes the security of the encryption system greatly improved by its nonlinear phase truncation. Deep learning (DL) as a method of machine learning was proposed decades ago. With the development of computer’s performance, the practicality of deep learning proves to be more and more obvious. Recently, deep learning has been effectively used in many fields such as biomedicine, object detection, etc. The good results have been achieved. In this article proposed is the attack to the PTFT encryption system by deep learning. Through the PTFT encryption system, we construct a plaintext-ciphertext paired image dataset and then train it by residual network (ResNet). There are two problems encountered by the traditional neural network model. One is vanishing or named exploding gradient, which makes training effect difficult to converge and the other is a degradation phenomenon. When continuing to increase the number of layers for a suitable depth model, the model accuracy will decline which is not caused by overfitting. This problem can be solved by the ResNet to a certain extent by directly bypassing and then taking the input information to the output to protect the integrity of the information. The biggest difference between ordinary directly connected convolutional neural networks and ResNet is that the ResNet has many bypass branches that directly connect the input to the subsequent layers, so that the subsequent layers can directly learn the residuals. The ResNet can automatically learn the decryption characteristics of the encryption system. Finally, the test set is used to test the decryption performance of the trained model. The data show that the model can restore the image with high quality and the model has a certain anti-noise ability. Compared with the two-step iterative amplitude recovery algorithm, the the method proposed in this paper can recover high quality image.

Telecentric in-line-and-off-axis hybrid digital holographic high-resolution reconstruction method

In-line digital holography usually employs a phase retrieval algorithm to decouple the phase information but fails to eliminate the unwanted DC and twin image terms when the measured sample does not agree with the sparsity. While the off-axis digital holography can efficiently remove the unwanted image terms but can not reserve the high frequencies of the sample to realize high resolution. The in-line-and-off-axis hybrid digital holography was then developed to provide a relatively high resolution digital holographic imaging without considering the effect of the unwanted terms. In other words, the in-line-and-off-axis hybrid digital holography merges all of the best virtues of the mentioned-above methods in an efficient and elegant way. However, this state-of-the-art method requires prior knowledge about the diffraction distance, which results in time-consuming and low accuracy. In other sense, telecentric technology can realize non-diffractive imaging without the knowledge about the diffraction distance or spherical aberration or defocusing aberration. Therefore, in this paper, a novel in-line-and-off-axis hybrid digital holography is proposed by introducing telecentric imaging architecture, and the corresponding reconstruction method is further proposed by utilizing constrained iterative approach. In this method, telecentric in-line-and-off-axis hybrid digital holography is first used to acquire focused off-axis and in-line holograms, respectively. The low resolution phase information is reconstructed from the off-axis hologram by using Fourier transform method with the help of the sample-free off-axis hologram, and then multiplexed with the amplitude information obtained from the in-line hologram to act as the initial complex amplitude in the iterative recovery process. As a result, constrained iterations are carried out in the spatial domain and frequency domain to realize high resolution and high speed reconstruction. After simulations, we build an experimental setup and demonstrate the operation of the method with USAF resolution target, onion cells and bee wings. Both the simulation and experimental results show that the proposed method can require no prior knowledge to suppress the phase disturbance caused by the unwanted image terms and optical aberrations, resulting in high speed and full utilization of spatial bandwidth product of the digital camera to yield high resolution reconstruction. We hope that the proposed method will have most practical applications in the case where large resolution, high speed and good quality are needed.

An Integral-Based Approach to Orthogonal AMP

Approximate message passing (AMP) is an iterative signal recovery algorithm for compressed sensing (CS) applications. In this letter, we present an integral-based orthogonal AMP (IB-OAMP) technique that avoids the requirements of AMP (and also the original form of OAMP) on differentiable and separable denoisers. The orthogonality in IB-OAMP can be established using a Monte Carlo method similar to the training stage in a machine-learning algorithm. These features make IB-OAMP attractive to be used in conjunction with some well-studied denoising algorithms.

Random Sampling With Iterative Recovery for Millimeter-Wave Imaging Systems

In this brief, an imaging methodology for mono-static millimeter-wave systems is introduced. When the measured object hologram is piece-wise smooth with sharp boundaries, both low-pass and high-pass components are present in a transform domain. Nevertheless, the edges are sparse in transform domain with respect to the image dimensions. It is shown that the down-sampling of the hologram with an appropriate rate determined by the sparsity level, includes the adequate information of all components for image reconstruction. Using random samples, an algorithm is proposed to recover all the sparse components, iteratively. Simulation results illustrate that a high-resolution recovery can be achieved. For instance, by using 20% of the randomly selected hologram samples, an image recovery with a PSNR of 20 dB is achieved and the object scan time is significantly reduced. The proposed methodology is applied to a practical imaging system operating at 30 GHz, where the results illustrate a successful reconstruction of the images.

Reduced complexity per-survivor iterative timing recovery using max-log-MAP algorithm

In this paper, we propose a reduced complexity (RC) per-survivor-processing (RC-PSP) iterative timing recovery scheme. The objective is to lessen the computational burden while approaching the optimal system performance of the existing full-complexity (FC) log-MAP (FC-PSP) iterative recovery scheme. This is achieved by utilising the max-log-MAP framework, which converts the multiplications and additions to maximum operations. According to Spasov, Gushev and Ristov, such framework consumes less power, reduces space on a chip and increases decoding speed. Robertson et al. showed that max-log-MAP PSP requires only 5 × 2M − 2 max operations, 10 × 2M additions and 8 multiplications by ±1. Furthermore, it offers a direct mathematical framework for converting the log-MAP-based PSP to max-log-MAP-based PSP for timing recovery. The simulations show that the proposed method outperforms the previous methods. The computational complexity for the max-log-MAP PSP is significantly reduced at the cost of a small bit error rate (BER) performance loss.