Iterative Recovery Algorithm Research Articles

Generalization error bounds for iterative recovery algorithms unfolded as neural networks

Abstract Motivated by the learned iterative soft thresholding algorithm (LISTA), we introduce a general class of neural networks suitable for sparse reconstruction from few linear measurements. By allowing a wide range of degrees of weight-sharing between the flayers, we enable a unified analysis for very different neural network types, ranging from recurrent ones to networks more similar to standard feedforward neural networks. Based on training samples, via empirical risk minimization, we aim at learning the optimal network parameters and thereby the optimal network that reconstructs signals from their low-dimensional linear measurements. We derive generalization bounds by analyzing the Rademacher complexity of hypothesis classes consisting of such deep networks, that also take into account the thresholding parameters. We obtain estimates of the sample complexity that essentially depend only linearly on the number of parameters and on the depth. We apply our main result to obtain specific generalization bounds for several practical examples, including different algorithms for (implicit) dictionary learning, and convolutional neural networks.

Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

Countless signal processing applications include the reconstruction of signals from few indirect linear measurements. The design of effective measurement operators is typically constrained by the underlying hardware and physics, posing a challenging and often even discrete optimization task. While the potential of gradient-based learning via the unrolling of iterative recovery algorithms has been demonstrated, it has remained unclear how to leverage this technique when the set of admissible measurement operators is structured and discrete. We tackle this problem by combining unrolled optimization with Gumbel reparametrizations, which enable the computation of low-variance gradient estimates of categorical random variables. Our approach is formalized by GLODISMO (Gradient-based Learning of DIscrete Structured Measurement Operators). This novel method is easy-to-implement, computationally efficient, and extendable due to its compatibility with automatic differentiation. We empirically demonstrate the performance and flexibility of GLODISMO in several prototypical signal recovery applications, verifying that the learned measurement matrices outperform conventional designs based on randomization as well as discrete optimization baselines.

Compressive sensing of ECG signals using plug-and-play regularization

Compressive Sensing (CS) has recently attracted attention for ECG data compression. In CS, an ECG signal is projected onto a small set of random vectors. Recovering the original signal from such compressed measurements remains a challenging problem. Traditional recovery methods are based on solving a regularized minimization problem, where a sparsity-promoting prior is used. In this paper, we propose an alternative iterative recovery algorithm based on the Plug-and-Play (PnP) method, which has recently become popular for imaging problems. In PnP, a powerful denoiser is used to implicitly perform regularization, instead of using hand-crafted regularizers; this has been found to be more successful than traditional methods. In this work, we use a PnP version of the Proximal Gradient Descent (PGD) algorithm for ECG recovery. To ensure mathematical convergence of the PnP algorithm, the signal denoiser in question needs to satisfy some technical conditions. We use a high-quality ECG signal denoiser fulfilling this condition by learning a Bayesian prior for small-sized signal patches. This guarantees that the proposed algorithm converges to a fixed point irrespective of the initialization. Importantly, through extensive experiments, we show that the reconstruction quality of the proposed method is superior to that of state-of-the-art methods.

A convergence analysis for iterative sparsification projection with soft-thresholding

The recently proposed iterative sparsification projection (ISP), a fast and robust sparse signal recovery algorithm framework, can be classified as smooth-ISP and nonsmooth-ISP. However, no convergence analysis has been established for the nonsmooth-ISP in the previous works. Motivated by this absence, the present paper provides a convergence analysis for ISP with soft-thresholding (ISP-soft) which is an instance of the nonsmooth-ISP. In our analysis, the composite operator of soft-thresholding and proximal projection is viewed as a fixed point mapping, whose nonexpansiveness plays a key role. Specifically, our convergence analysis for the sequence generated by ISP-soft can be summarized as follows: 1) For each inner loop, we prove that the sequence has a unique accumulation point which is a fixed point, and show that it is a Cauchy sequence; 2) for the last inner loop, we prove that the accumulation point of the sequence is a critical point of the objective function if the final value of the threshold satisfies a condition, and show that the corresponding objective values are monotonically nonincreasing. A numerical experiment is given to validate some of our results and intuitively present the convergence.

Attacking asymmetric cryptosystem based on phase truncated Fourier fransform by deep learning

Most of optical encryption systems are symmetric cryptosystems. The plaintext and the ciphertext in optical image encryption are related linearly. The security of the system needs to be strengthened. The asymmetric cryptosystem based on phase truncated Fourier transforms (PTFT) makes the security of the encryption system greatly improved by its nonlinear phase truncation. Deep learning (DL) as a method of machine learning was proposed decades ago. With the development of computer’s performance, the practicality of deep learning proves to be more and more obvious. Recently, deep learning has been effectively used in many fields such as biomedicine, object detection, etc. The good results have been achieved. In this article proposed is the attack to the PTFT encryption system by deep learning. Through the PTFT encryption system, we construct a plaintext-ciphertext paired image dataset and then train it by residual network (ResNet). There are two problems encountered by the traditional neural network model. One is vanishing or named exploding gradient, which makes training effect difficult to converge and the other is a degradation phenomenon. When continuing to increase the number of layers for a suitable depth model, the model accuracy will decline which is not caused by overfitting. This problem can be solved by the ResNet to a certain extent by directly bypassing and then taking the input information to the output to protect the integrity of the information. The biggest difference between ordinary directly connected convolutional neural networks and ResNet is that the ResNet has many bypass branches that directly connect the input to the subsequent layers, so that the subsequent layers can directly learn the residuals. The ResNet can automatically learn the decryption characteristics of the encryption system. Finally, the test set is used to test the decryption performance of the trained model. The data show that the model can restore the image with high quality and the model has a certain anti-noise ability. Compared with the two-step iterative amplitude recovery algorithm, the the method proposed in this paper can recover high quality image.

An Integral-Based Approach to Orthogonal AMP

Approximate message passing (AMP) is an iterative signal recovery algorithm for compressed sensing (CS) applications. In this letter, we present an integral-based orthogonal AMP (IB-OAMP) technique that avoids the requirements of AMP (and also the original form of OAMP) on differentiable and separable denoisers. The orthogonality in IB-OAMP can be established using a Monte Carlo method similar to the training stage in a machine-learning algorithm. These features make IB-OAMP attractive to be used in conjunction with some well-studied denoising algorithms.

MH-Net: A Learnable Multi-Hypothesis Network for Compressed Video Sensing

This paper establishes a novel recovery network called MH-Net-a framework for compressed video sensing (CVS) based on recently emerging deep neural networks (DNNs) techniques. MH-Net exploits temporal correlation between frames in the form of multi-hypothesis (MH) prediction, and learns a high-dimensional domain which is more robust for prediction generation. After the MH prediction, a special residual network is used in MH-Net to reconstruct the residuals between the MH prediction and the desired frame from their measurements. The final reconstruction is derived by adding the reconstructed residuals to the MH prediction. Unlike the block-wise reconstruction in existing DNN-based CS architecture, MH-Net builds a mapping from block measurements to a complete frame reconstruction, leading to better reconstruction quality. Benefitting from the DNN's nature, the forward propagation of MH-Net is extremely fast, making it suitable for real-time applications. Experimental results show that MH-Net presents a better recovery performance compared with existing DNN-based recovery methods and traditional iterative recovery algorithms.

Separable Reversible Data Hiding in Encrypted JPEG Bitstreams

While most techniques of reversible data hiding in encrypted images (RDH-EI) are developed for uncompressed images, this paper provides a separable reversible data hiding protocol for encrypted JPEG bitstreams. We first propose a JPEG encryption algorithm, which enciphers an image to a smaller size and keeps the format compliant to JPEG decoder. After a content owner uploads the encrypted JPEG bitstream to a remote server, a data hider embeds an additional message into the encrypted copy without changing the bitstream size. On the recipient side, the original bitstream can be reconstructed losslessly using an iterative recovery algorithm based on the blocking artifact. Since message extraction and image recovery are separable, anyone who has the embedding key can extract the message from the marked encrypted copy. Experimental results show that the proposed method outperforms a previous work in terms of separation capability, embedding capacity and security.

Single-shot quantitative phase microscopy based on color-multiplexed Fourier ptychography.

We present a single-shot quantitative phase imaging (QPI) method based on color-multiplexed Fourier ptychographic microscopy (FPM). Three light-emitting diode (LED) elements with respective R/G/B channels in a programmable LED array illuminate the specimen simultaneously, providing triangle oblique illuminations matching the numerical aperture of the objective precisely. A color image sensor records the light transmitted through the specimen, and three monochromatic intensity images at each color channel are then separated and utilized to recover the phase of the specimen. After one-step deconvolution based on the phase contrast transfer function, the obtained initial phase map is further refined by the FPM-based iterative recovery algorithm to overcome pixel-aliasing and improve the phase recovery accuracy. The high-speed, high-throughput QPI capabilities of the proposed approach are demonstrated by imaging HeLa cells mitosis in vitro, achieving a half-pitch resolution of 388nm across a wide field of view of 1.33 mm2 at camera-limited frame rates (50fps).

See-through Detection and 3D Reconstruction Using Terahertz Leaky-Wave Radar Based on Sparse Signal Processing

We experimentally demonstrate see-through detection and 3D reconstruction using terahertz leaky-wave radar based on sparse signal processing. The application of terahertz waves to radar has received increasing attention in recent years for its potential to high-resolution and see-through detection. Among others, the implementation using a leaky-wave antenna is promising for compact system integration with beam steering capability based on frequency sweep. However, the use of a leaky-wave antenna poses a challenge on signal processing. Since a leaky-wave antenna combines the entire signal captured by each part of the aperture into a single output, the conventional array signal processing assuming access to a respective antenna element is not applicable. In this paper, we apply an iterative recovery algorithm “CoSaMP” to signals acquired with terahertz leaky-wave radar for clutter mitigation and aperture synthesis. We firstly demonstrate see-through detection of target location even when the radar is covered with an opaque screen, and therefore, the radar signal is disturbed by clutter. Furthermore, leveraging the robustness of the algorithm against noise, we also demonstrate 3D reconstruction of distributed targets by synthesizing signals collected from different orientations. The proposed approach will contribute to the smart implementation of terahertz leaky-wave radar.

Efficient protocol header recovery algorithm based on iterative decoding between layers

The iterative header recovery algorithm holds the potential of improving the performance of wireless communication, yet it has to face the challenge of reducing the high complexity. In this study, a method of constructing a candidate set around the observation sequence is proposed to approximate the original code space, by exploiting the channel measurement information. This approximation method narrows the decoding processing scope resulting in an effective estimation of the extrinsic information and a significant reduction of the computational complexity. The extrinsic information in soft form is derived to exploit the soft input and promote header recovery performance. Simulation results and analysis show that the approximation method not only has a comparable performance, but also decreases the computational complexity and the storage consumption, which are of great importance of practical application.

$L_{1}$ -Regularization-Based SAR Imaging and CFAR Detection via Complex Approximated Message Passing

Synthetic aperture radar (SAR) is a widely used active high-resolution microwave imaging technique that has all-time and all-weather reconnaissance ability. Compared with traditionally matched filtering (MF)-based methods, ${L_{q}}({0 \le q \le 1})$ regularization technique can efficiently improve SAR imaging performance e.g., suppressing sidelobes and clutter. However, conventional $L_{q}$ -regularization-based SAR imaging approach requires transferring the 2-D echo data into a vector and reconstructing the scene via 2-D matrix operations. This leads to significantly more computational complexity compared with MF, and makes it very difficult to apply in high-resolution and wide-swath imaging. Typical $L_{q}$ regularization recovery algorithms, e.g., iterative thresholding algorithm, can improve imaging performance of bright targets, but not preserve the image background distribution well. Thus, image background statistical-property-based applications, such as constant false alarm rate (CFAR) detection, cannot be applied to regularization recovered SAR images. On the other hand, complex approximated message passing (CAMP), an iterative recovery algorithm for $L_{1}$ regularization reconstruction, can achieve not only the sparse estimation of the original signal as typical regularization recovery algorithms but also a nonsparse solution simultaneously. In this paper, two novel CAMP-based SAR imaging algorithms are proposed for raw data and complex radar image data, respectively, along with CFAR detection via the CAMP recovered nonsparse result. The proposed method for raw data can not only improve SAR image performance as conventional $L_{1}$ regularization technique but also reduce the computational cost efficiently. While only when we have MF recovered SAR complex image rather than raw data, the proposed method for complex image data can achieve a similar reconstructed image quality as the regularization-based SAR imaging approach using the full raw data. The most important contribution of this paper is that the proposed CAMP-based methods make CFAR detection based on the regularization reconstruction SAR image possible using their nonsparse scene estimations, which has a similar background statistical distribution as the MF recovered images. The experimental results validated the effectiveness of the proposed methods and the feasibility of the recovered nonsparse images being used for CFAR detection.

Peeling Decoding of LDPC Codes with Applications in Compressed Sensing

We present a new approach for the analysis of iterative peeling decoding recovery algorithms in the context of Low-Density Parity-Check (LDPC) codes and compressed sensing. The iterative recovery algorithm is particularly interesting for its low measurement cost and low computational complexity. The asymptotic analysis can track the evolution of the fraction of unrecovered signal elements in each iteration, which is similar to the well-known density evolution analysis in the context of LDPC decoding algorithm. Our analysis shows that there exists a threshold on the density factor; if under this threshold, the recovery algorithm is successful; otherwise it will fail. Simulation results are also provided for verifying the agreement between the proposed asymptotic analysis and recovery algorithm. Compared with existing works of peeling decoding algorithm, focusing on the failure probability of the recovery algorithm, our proposed approach gives accurate evolution of performance with different parameters of measurement matrices and is easy to implement. We also show that the peeling decoding algorithm performs better than other schemes based on LDPC codes.

Image Reconstruction Based on Compressive Sensing via CoSaMP

Compressive Sampling Matching Pursuit (CoSaMP) is a new iterative recovery algorithm which has splendid theoretical guarantees for convergence and delivers the same guarantees as the best optimization-based approaches. In this paper, we propose a new signal recovery framework which combines CoSaMP and Curvelet transform for better performance. In classic CoSaMP, the number of iterations is fixed. We discuss a new stopping rule to halting the algorithm in this paper. In addition, the choice of several adjustable parameters in algorithm such as the number of measurements and the sparse level of the signal also will impact the performance. So we gain above parameters via a large number of experiments. According to experiments, we determine an optimum value for the parameters to use in this application. The experiments show that the new method not only has better recovery quality and higher PSNRs, but also can achieve optimization steadily and effectively.

Signal Space CoSaMP for Sparse Recovery With Redundant Dictionaries

Compressive sensing (CS) has recently emerged as a powerful framework for acquiring sparse signals. The bulk of the CS literature has focused on the case where the acquired signal has a sparse or compressible representation in an orthonormal basis. In practice, however, there are many signals that cannot be sparsely represented or approximated using an orthonormal basis, but that do have sparse representations in a redundant dictionary. Standard results in CS can sometimes be extended to handle this case provided that the dictionary is sufficiently incoherent or well conditioned, but these approaches fail to address the case of a truly redundant or overcomplete dictionary. In this paper, we describe a variant of the iterative recovery algorithm CoSaMP for this more challenging setting. We utilize the \mbi D-RIP, a condition on the sensing matrix analogous to the well-known restricted isometry property. In contrast to prior work, the method and analysis are “signal-focused”; that is, they are oriented around recovering the signal rather than its dictionary coefficients. Under the assumption that we have a near-optimal scheme for projecting vectors in signal space onto the model family of candidate sparse signals, we provide provable recovery guarantees. Developing a practical algorithm that can provably compute the required near-optimal projections remains a significant open problem, but we include simulation results using various heuristics that empirically exhibit superior performance to traditional recovery algorithms.

Iterative algorithms for total variation-like reconstructions in seismic tomography

A qualitative comparison of total variation like penalties (total variation, Huber variant of total variation, total generalized variation, ...) is made in the context of global seismic tomography. Both penalized and constrained formulations of seismic recovery problems are treated. A number of simple iterative recovery algorithms applicable to these problems are described. The convergence speed of these algorithms is compared numerically in this setting. For the constrained formulation a new algorithm is proposed and its convergence is proven.

CoSaMP: Iterative signal recovery from incomplete and inaccurate samples

Compressive sampling offers a new paradigm for acquiring signals that are compressible with respect to an orthonormal basis. The major algorithmic challenge in compressive sampling is to approximate a compressible signal from noisy samples. This paper describes a new iterative recovery algorithm called CoSaMP that delivers the same guarantees as the best optimization-based approaches. Moreover, this algorithm offers rigorous bounds on computational cost and storage. It is likely to be extremely efficient for practical problems because it requires only matrix–vector multiplies with the sampling matrix. For compressible signals, the running time is just O ( N log 2 N ) , where N is the length of the signal.

Iterative image processing using a cavity with a phase-conjugate mirror

We describe an optical image processing system with regenerative feedback that utilizes a cavity with a phase-conjugate mirror. The system's characteristics and limitations are discussed, and its application to the implementation of iterative recovery algorithms (e.g., Gerchberg algorithm) is considered and demonstrated experimentally.