Sparse Signal Approximation Research Articles

A Signal Dependent Analysis of Orthogonal Least Squares Based on the Restricted Isometry Property

The Orthogonal Least Squares (OLS) is a widespread greedy algorithm for sparse signal approximation or reconstruction. We here address the problem of sufficient condition for exact signal support recovery via OLS. Based on the generalized Wielandt inequality, new bounds on the OLS selection rule are provided that enable a simple approach of the Restricted Isometry Property based analysis of OLS. Since these new bounds naturally take the signal magnitude distribution into account, they allow a derivation of a new relaxed exact support recovery condition in the noise-free case. Moreover, we study their influence on the design of such new conditions in bounded noise and Gaussian noise cases.

Reversible image authentication scheme with blind content reconstruction based on compressed sensing

Reversible image authentication (RIA) schemes embed fragile watermarks to protect images against tampering attacks. If a marked image is not attacked, watermarks can be completely extracted and eliminated to obtain the original cover image. Tampered image regions can be reconstructed by including recovery information as part of the watermarks. In this paper, we introduce a novel blind RIA scheme for tampering localization and reconstruction. The proposed scheme is based on compressed sensing technique to obtain reference values, which are embedded into the cover image to perform sparse signal reconstruction using only the embedded information. Signal sparsity is enhanced in the discrete cosine transform (DCT) domain by omitting some high-frequency coefficients. Experimental results indicate that better sparse signal approximations can be obtained when some DCT coefficients are omitted, improving image reconstructions. The proposed scheme outperforms similar state-of-the-art works when reconstructing tampered image regions, especially when the damaged area size is increased.

Tangent vector-based gradient method with [formula omitted]-regularization: Iterative half thresholding algorithm for CS-MRI

Object: This paper presents a new method using tangent vector-based l12-regularization for compressed sensing MR image reconstruction. Materials and MethodsThe proposed method with l12-regularization is tested on four datasets: (i) 1-D sparse signal (ii) numerical cardiac phantom, (iii & iv) two sets of in-vivo cardiac MRI datasets acquired using 30 receiver coil elements with Cartesian and radial trajectories on 3T scanner. The results are compared with standard CS reconstruction, which utilizes l1-regularization. The experiments were also conducted for two different types of samplings: (i) cartesian sub-sampling and (ii) 2D random Gaussian sub-sampling. ResultsThe quality of the reconstructed images is validated through Root Mean Square Error (RMSE) and Peak Signal-to-Noise Ratio (PSNR). The results show that the proposed method outperforms the standard CS reconstructions in our experiments with an improvement of 54.8% in RMSE and 14.3% in terms of PSNR. Moreover, the Gaussian random sub-sampling-based image reconstruction results are better than the Cartesian sub-sampling-based reconstruction results. ConclusionThe results show that the proposed method yields a good sparse signal approximation and superior convergence behavior, which implies a promising technique for the reconstruction of cardiac MR images as compared to the conventional CS algorithm.

Newton-Step-Based Hard Thresholding Algorithms for Sparse Signal Recovery

Sparse signal recovery or compressed sensing can be formulated as certain sparse optimization problems. The classic optimization theory indicates that the Newton-like method often has a numerical advantage over the gradient method for nonlinear optimization problems. In this paper, we propose the so-called Newton-step-based iterative hard thresholding (NSIHT) and the Newton-step-based hard thresholding pursuit (NSHTP) algorithms for sparse signal recovery and signal approximation. Different from the traditional iterative hard thresholding (IHT) and hard thresholding pursuit (HTP), the proposed algorithms adopts the Newton-like search direction instead of the steepest descent direction. A theoretical analysis for the proposed algorithms is carried out, and some sufficient conditions for the guaranteed success of sparse signal recovery via these algorithms are established. Our results are shown under the restricted isometry property which is one of the standard assumptions widely used in the field of compressed sensing and signal approximation. The empirical results obtained from synthetic data recovery indicate that the proposed algorithms are efficient signal recovery methods. The numerical stability of our algorithms in terms of the residual reduction is also investigated through simulations.

Stabilized Stepwise Orthogonal Matching Pursuit for Sparse Signal Approximation

Orthogonal Matching Pursuit (OMP) algorithm is equipped with the capability to decompose any signal into a linear expansion of waveforms, which are selected from a redundant functional dictionary. Nevertheless, classical OMP algorithm suffers a heavy computational burden due to its single element selection strategy in each repetition. Recently, an accelerated implementation called stage wise orthogonal matching pursuit (StOMP) algorithm has been proposed through exploiting a multiple elements selection scheme based on an iterative threshold. However, as the defined threshold is a function of an empirical and undetermined parameter, such a reconstruction scheme is not optimal and the algorithm may get obstructed in some specific conditions. This manuscript presents an adaptive threshold selection strategy which takes signal structure into consideration and furthermore, a regularized iterative framework for sparse signal approximation is suggested. Compared with classical StOMP approach, these efforts can provide robust and more attractive approximation performance for sparse signal recoveries. Experimental results present the substantial improvements of these optimizations.

Online feature learning for condition monitoring of rotating machinery

Condition-based maintenance of rotating machinery requires efficient condition monitoring methods that enable early detection of abnormal operational conditions and faults. This is a challenging problem because machines are different and change characteristics over time due to wear and maintenance. The efficiency and scalability of conventional condition monitoring methods are limited by the need for manual analysis and re-configuration. The problem to extract relevant features from condition monitoring signals and thereby detect and analyze changes in such signals is a central issue, which in principle can be addressed using machine learning methods. Former work demonstrates that dictionary learning can be used to automatically derive signal features that characterize different operational conditions and faults of a rotating machine, but the use of such methods for online condition monitoring purposes is an open problem. Here we investigate online learning of features using dictionary learning. We describe dictionary distance and signal fidelity based heuristics for anomaly detection, and we study the time-propagated features and sparse approximation of vibration and acoustic emission signals in three different case studies. We present results of numerical experiments with different hyperparameters affecting the approximation accuracy, computational cost, and the adaptation rate of the learned features. We find that the learned features change slowly under normal variations of the operational conditions in comparison to the rapid adaptation observed when a fault appears (bearing defects, magnetite particles in the lubricant, or plastic deformation of steel). Furthermore, we find that a sparse signal approximation with 2.5% preserved coefficients based on a propagated dictionary is sufficient for bearing defect detection.

Sparse Signal Approximation via Nonseparable Regularization

The calculation of a sparse approximate solution to a linear system of equations is often performed using either L1-norm regularization and convex optimization or nonconvex regularization and nonconvex optimization. Combining these principles, this paper describes a type of nonconvex regularization that maintains the convexity of the objective function, thereby allowing the calculation of a sparse approximate solution via convex optimization. The preservation of convexity is viable in the proposed approach because it uses a regularizer that is nonseparable. The proposed method is motivated and demonstrated by the calculation of sparse signal approximation using tight frames. Examples of denoising demonstrate improvement relative to L1 norm regularization.

Iterative Sparsification-Projection: Fast and Robust Sparse Signal Approximation

In this paper, we address recovery of sparse signals from compressed measurements, and sparse signal approximation, which have received considerable attention over the last decade. First, we revisit smoothed L0 (SL0), a well-known sparse recovery algorithm, and give some insights into it that have not been noticed previously. Specifically, we re-derive the SL0 algorithm based on proximal methods, and using recent results in solving nonconvex problems by proximal algorithms, we provide a convergence guarantee for it. In addition, inspired by this derivation, we propose a general family of algorithms, which we call iterative sparsification-projection (ISP), having SL0 as a special member. Our algorithmic framework starts with an initial guess for the unknown sparse vector, and then iteratively sparsifies it (using a fixed threshold) followed by projecting the result onto the admissible solution set. The threshold is then decreased, and the same process is repeated. The algorithm terminates when the threshold becomes sufficiently small, or another stopping criterion is satisfied. We also propose a robust projection to handle the situations with observation noise or model uncertainties. Our extensive simulations confirm the promising performance of the ISP algorithms compared with some well-known algorithms.

Pseudo-inverses of difference matrices and their application to sparse signal approximation

We derive new explicit expressions for the components of Moore–Penrose inverses of symmetric difference matrices. These generalized inverses are applied in a new regularization approach for scattered data interpolation based on partial differential equations. The columns of the Moore–Penrose inverse then serve as elements of a dictionary that allow a sparse signal approximation. In order to find a set of suitable data points for signal representation we apply the orthogonal matching pursuit (OMP) method.

Sparse Modeling with Applications to Speech Processing: A Survey

<p>Nowadays, there has been a growing interest in the study of sparse approximation of signals. Using an over-complete dictionary consisting of prototype signals or atoms, signals are described by sparse linear combinations of these atoms. Applications that use sparse representation are many and include compression, source separation, enhancement, and regularization in inverse problems, feature extraction, and more. This article introduces a literature review of sparse coding applications in the field of speech processing.</p>

Union of low‐rank subspaces detector

The problem of signal detection using a flexible and general model is considered. Owing to applicability and flexibility of sparse signal representation and approximation, it has attracted a lot of attention in many signal processing areas. In this study, the authors propose a new detection method based on sparse decomposition in a union of subspaces model. Their proposed detector uses a dictionary that can be interpreted as a bank of matched subspaces. This improves the performance of signal detection, as it is a generalisation for detectors. Low-rank assumption for the desired signals implies that the representations of these signals in terms of some proper bases would be sparse. Their proposed detector exploits sparsity in its decision rule. They demonstrate the high efficiency of their method in the cases of voice activity detection in speech processing.

Multivariate [formula omitted]-molecules

The suboptimal performance of wavelets with regard to the approximation of multivariate data gave rise to new representation systems, specifically designed for data with anisotropic features. Some prominent examples of these are given by ridgelets, curvelets, and shearlets, to name a few.The great variety of such so-called directional systems motivated the search for a common framework, which unites many under one roof and enables a simultaneous analysis, for example with respect to approximation properties. Building on the concept of parabolic molecules, the recently introduced framework of α-molecules does in fact include the previous mentioned systems. Until now however it is confined to the bivariate setting, whereas nowadays one often deals with higher dimensional data. This motivates the extension of this unifying theory to dimensions larger than 2, put forward in this work. In particular, we generalize the central result that the cross-Gramian of any two systems of α-molecules will to some extent be localized.As an exemplary application, we investigate the sparse approximation of video signals, which are instances of 3D data. The multivariate theory allows us to derive almost optimal approximation rates for a large class of representation systems.

Impulse feature extraction method for machinery fault detection using fusion sparse coding and online dictionary learning

Impulse components in vibration signals are important fault features of complex machines. Sparse coding (SC) algorithm has been introduced as an impulse feature extraction method, but it could not guarantee a satisfactory performance in processing vibration signals with heavy background noises. In this paper, a method based on fusion sparse coding (FSC) and online dictionary learning is proposed to extract impulses efficiently. Firstly, fusion scheme of different sparse coding algorithms is presented to ensure higher reconstruction accuracy. Then, an improved online dictionary learning method using FSC scheme is established to obtain redundant dictionary and it can capture specific features of training samples and reconstruct the sparse approximation of vibration signals. Simulation shows that this method has a good performance in solving sparse coefficients and training redundant dictionary compared with other methods. Lastly, the proposed method is further applied to processing aircraft engine rotor vibration signals. Compared with other feature extraction approaches, our method can extract impulse features accurately and efficiently from heavy noisy vibration signal, which has significant supports for machinery fault detection and diagnosis.

Broadband acoustic-source localization using passive sonar via multitask learning

Passive sonar is an attractive technology for underwater acoustic-source localization that enables the localization system to conceal its presence and does not perturb the maritime environment. Notwithstanding its appeal, passive-sonar-based localization is challenging due to the complexities of underwater acoustic propagation. Different from alternatives based on matched-field processing whose localization performance severely deteriorate when localizing multiple sources and when faced with model mismatch, this work casts the broadband underwater acoustic-source localization problem as a multitask learning (MTL) problem, thereby enabling robust and high-resolution localization. Here, each task refers to a sparse signal approximation problem over a single frequency. MTL provides an elegant framework for exchanging information across the individual regression problems and constructing an aggregate (across frequencies) source localization map. The localization problem is formulated as a stochastic least-squares optimization problem with a group sparsity constraint enforcing a common support across frequency maps. Efficient algorithms based on block coordinate descent are developed for solving the localization problem. Predictor screening rules are also developed to further reduce the computational complexity of the proposed method. Numerical tests on real data illustrate and compare the localization performance of the proposed algorithm to that of competitive alternatives.

Learning Parametric Dictionaries for Signals on Graphs

In sparse signal representation, the choice of a dictionary often involves a tradeoff between two desirable properties – the ability to adapt to specific signal data and a fast implementation of the dictionary. To sparsely represent signals residing on weighted graphs, an additional design challenge is to incorporate the intrinsic geometric structure of the irregular data domain into the atoms of the dictionary. In this work, we propose a parametric dictionary learning algorithm to design data-adapted, structured dictionaries that sparsely represent graph signals. In particular, we model graph signals as combinations of overlapping local patterns. We impose the constraint that each dictionary is a concatenation of subdictionaries, with each subdictionary being a polynomial of the graph Laplacian matrix, representing a single pattern translated to different areas of the graph. The learning algorithm adapts the patterns to a training set of graph signals. Experimental results on both synthetic and real datasets demonstrate that the dictionaries learned by the proposed algorithm are competitive with and often better than unstructured dictionaries learned by state-of-the-art numerical learning algorithms in terms of sparse approximation of graph signals. In contrast to the unstructured dictionaries, however, the dictionaries learned by the proposed algorithm feature localized atoms and can be implemented in a computationally efficient manner in signal processing tasks such as compression, denoising, and classification.

A Multilevel Iterated-Shrinkage Approach to $l_{1}$ Penalized Least-Squares Minimization

The area of sparse approximation of signals is drawing tremendous attention in recent years. Typically, sparse solutions of underdetermined linear systems of equations are required. Such solutions are often achieved by minimizing an <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> penalized least squares functional. Various iterative-shrinkage algorithms have recently been developed and are quite effective for handling these problems, often surpassing traditional optimization techniques. In this paper, we suggest a new iterative multilevel approach that reduces the computational cost of existing solvers for these inverse problems. Our method takes advantage of the typically sparse representation of the signal, and at each iteration it adaptively creates and processes a hierarchy of lower-dimensional problems employing well-known iterated shrinkage methods. Analytical observations suggest, and numerical results confirm, that this new approach may significantly enhance the performance of existing iterative shrinkage algorithms in cases where the matrix is given explicitly.

SSIM-inspired image restoration using sparse representation

Recently, sparse representation based methods have proven to be successful towards solving image restoration problems. The objective of these methods is to use sparsity prior of the underlying signal in terms of some dictionary and achieve optimal performance in terms of mean-squared error, a metric that has been widely criticized in the literature due to its poor performance as a visual quality predictor. In this work, we make one of the first attempts to employ structural similarity (SSIM) index, a more accurate perceptual image measure, by incorporating it into the framework of sparse signal representation and approximation. Specifically, the proposed optimization problem solves for coefficients with minimum norm and maximum SSIM index value. Furthermore, a gradient descent algorithm is developed to achieve SSIM-optimal compromise in combining the input and sparse dictionary reconstructed images. We demonstrate the performance of the proposed method by using image denoising and super-resolution methods as examples. Our experimental results show that the proposed SSIM-based sparse representation algorithm achieves better SSIM performance and better visual quality than the corresponding least square-based method.

Kronecker Compressive Sensing

Compressive sensing (CS) is an emerging approach for the acquisition of signals having a sparse or compressible representation in some basis. While the CS literature has mostly focused on problems involving 1-D signals and 2-D images, many important applications involve multidimensional signals; the construction of sparsifying bases and measurement systems for such signals is complicated by their higher dimensionality. In this paper, we propose the use of Kronecker product matrices in CS for two purposes. First, such matrices can act as sparsifying bases that jointly model the structure present in all of the signal dimensions. Second, such matrices can represent the measurement protocols used in distributed settings. Our formulation enables the derivation of analytical bounds for the sparse approximation of multidimensional signals and CS recovery performance, as well as a means of evaluating novel distributed measurement schemes.

A matching pursuit method for approximating overlapping ultrasonic echoes

Ultrasonic pulse-echo methods have been used extensively in measuring the thickness of layered structures as well as those of thin adhesive interface layers. When acoustically measuring thin layers, the resulting echoes from two successive interfaces overlap in time, limiting the minimum thickness that can be resolved using conventional pulse-echo techniques. In this paper, we propose a method, named support matching pursuit (SMP), for resolving the individual echoes. The method is based on the concept of sparse signal approximation in an overcomplete dictionary composed of Gabor atoms (elementary functions). Although the dictionary enables highly flexible approximations, it is also overcomplete, which implies that the approximation is not unique. We propose a method for approximation in which each ultrasonic echo is principally represented by a single atom and therefore has a physical interpretation. SMP operates similarly to the sparse matching pursuit (MP) method. It iteratively improves the approximation by adding, at each iteration, a single atom to the solution set. However, our atom selection criterion utilizes the time localization nature of ultrasonic echoes, which causes portions of a multi-echo ultrasonic signal to be composed mainly from a single echo. This leads to accurate approximations in which each echo is characterized by a set of physical parameters that represent the composing ultrasonic echoes. In the current research we compare SMP to other sparse approximation methods such as MP and basis pursuit (BP). We perform simulations and experiments on adhesively bonded structures which clearly demonstrate the superior performance of the SMP method over the MP and BP methods.

Matching-Pursuits Dissimilarity Measure for Shape-Based Comparison and Classification of High-Dimensional Data

In this paper, a new matching pursuits dissimilarity measure (MPDM) is presented that compares two signals using the information provided by their matching pursuits (MP) approximations, without requiring any prior domain knowledge. MPDM is a flexible and differentiable measure that can be used to perform shape-based comparisons and fuzzy clustering of very high-dimensional, possibly compressed, data. A novel prototype based classification algorithm, which is termed the computer aided minimization procedure (CAMP), is also proposed. The CAMP algorithm uses the MPDM with the competitive agglomeration (CA) fuzzy clustering algorithm to build reliable shape based prototypes for classification. MP is a well known sparse signal approximation technique, which is commonly used for video and image coding. The dictionary and coefficient information produced by MP has previously been used to define features to build discrimination and prototype based classifiers. However, existing MP based classification applications are quite problem domain specific, thus making their generalization to other problems quite difficult. The proposed CAMP algorithm is the first MP based classification system that requires no assumptions about the problem domain and builds a bridge between the MP and fuzzy clustering algorithms. Experimental results also show that the CAMP algorithm is more resilient to outliers in test data than the multilayer perceptron (MLP) and support-vector-machine (SVM) classifiers, as well as prototype-based classifiers using the Euclidean distance as their dissimilarity measure.