Sparse Representation Problem Research Articles

High Resolution Ranging with Small Sample Number under Low SNR Utilizing RIP-OMCS Strategy and AHRC l1 Minimization for Laser Radar

This manuscript presents a novel scheme to achieve high-resolution laser-radar ranging with a small sample number under low signal-to-noise ratio (SNR) conditions. To reduce the sample number, the Restricted Isometry Property-based optimal multi-channel coprime-sampling (RIP-OMCS) strategy is established. In the RIP-OMCS strategy, the data collected across multiple channels with very low coprime-sampling rates can record accurate range information on each target. Further, the asynchronous problem caused by channel sampling-time errors is considered. The sampling-time errors are estimated using the cross-correlation function. After canceling the asynchronous problem, the data collected by multiple channels are then merged into non-uniform sampled signals. Using data combination, target-range estimation is converted into an optimization problem of sparse representation consisting of a non-uniform Fourier dictionary. This optimization problem is solved using adaptive hybrid re-weighted constraint (AHRC) l1 minimization. Two constraints are formed from statistical attributes of the targets and clutter. Moreover, as the detailed characteristics of the target, clutter, and noise are unknown before the solution, the two constraints can be adaptively modified, which guarantees that l1 minimization obtains the high-resolution range profile and accurate distance of all targets under a low SNR. Our experiments confirmed the effectiveness of the proposed method.

Low-Shot Unsupervised Visual Anomaly Detection via Sparse Feature Representation.

Visual anomaly detection is an essential component in modern industrial manufacturing. Existing studies using notions of pairwise similarity distance between a test feature and nominal features have achieved great breakthroughs. However, the absolute similarity distance lacks certain generalizations, making it challenging to extend the comparison beyond the available samples. This limitation could potentially hamper anomaly detection performance in scenarios with limited samples. This article presents a novel sparse feature representation anomaly detection (SFRAD) framework, which formulates the anomaly detection as a sparse feature representation problem; and notably proposes an anomaly score by orthogonal matching pursuit (ASOMP) as a novel detection metric. Specifically, SFRAD calculates the Gaussian kernel distance between the test feature and its sparse representation in the nominal feature space for anomaly detection. Here, the orthogonal matching pursuit (OMP) algorithm is adopted to achieve the sparse feature representation. Moreover, to construct a low-redundancy memory bank storing the basis features for sparse representation, a novel basis feature sampling (BFS) algorithm is proposed by considering both the maximum coverage and the optimum feature representation simultaneously. As a result, SFRAD incorporates both the advantages of absolute similarity and linear representation; and this enhances the generalization in low-shot scenarios. Extensive experiments on the MVTec anomaly detection (MVTec AD), Kolektor surface-defect dataset (KolektorSDD), Kolektor surface-defect dataset 2 (KolektorSDD2), MVTec logical constraints anomaly detection (MVTec LOCO AD), Visual anomaly (VISA), Modified national institute of standards and technology (MNIST), and CIFAR-10 datasets demonstrate that our proposed SFRAD outperforms the previous methods and achieves state-of-the-art unsupervised anomaly detection performance. Notably, significantly improved outcomes and results have also been achieved on low-shot anomaly detection. Code is available at https://github.com/fanghuisky/SFRAD.

Non-contact heart rate measurement from face video based on group sparse representation

Non-contact heart rate measurement based on face video is rapidly developed due to its comfort and wide application. However, it is difficult to extract the pulse signals for non-contact heart rate measurement due to the various interference factors, such as illumination variation, head motion and face expression. In this paper, we propose group sparse representation to reconstruct the pulse signals, then estimate the heart rate based on the fact that the real heart rate is consistent at the same time from different sub-regions. Specifically, we formulate the reconstruction of pulse signals as group sparse representation problem and require the raw signals of all the sub-regions to be similar sparse representation. Firstly, we use the face detection algorithm to obtain the region of interest (ROI) and divide it into sub-regions, followed by the distortion compensation of color signals from different sub-regions. Then we extract the sub-region chrominance signals and select the high-quality chrominance signals to construct the raw pulse signals matrix. After that, we construct a mixed pulse dictionary containing discrete cosine bases and wavelet bases considering the periodicity and pulsatility of the pulse signals. Finally, we conduct group sparse representation to reconstruct the pulse signals and estimate the heart rate via spectral analysis based on the reconstructed pulse signals. Experimental results on three public datasets show that this method outperforms most existing heart rate measurement methods.

Design of feature selection algorithm for high-dimensional network data based on supervised discriminant projection.

High dimension and complexity of network high-dimensional data lead to poor feature selection effect network high-dimensional data. To effectively solve this problem, feature selection algorithms for high-dimensional network data based on supervised discriminant projection (SDP) have been designed. The sparse representation problem of high-dimensional network data is transformed into an Lp norm optimization problem, and the sparse subspace clustering method is used to cluster high-dimensional network data. Dimensionless processing is carried out for the clustering processing results. Based on the linear projection matrix and the best transformation matrix, the dimensionless processing results are reduced by combining the SDP. The sparse constraint method is used to achieve feature selection of high-dimensional data in the network, and the relevant feature selection results are obtained. The experimental findings demonstrate that the suggested algorithm can effectively cluster seven different types of data and converges when the number of iterations approaches 24. The F1 value, recall, and precision are all kept at high levels. High-dimensional network data feature selection accuracy on average is 96.9%, and feature selection time on average is 65.1 milliseconds. The selection effect for network high-dimensional data features is good.

Unconstrained neighbor selection for minimum reconstruction error-based K-NN classifiers

It is essential to define more convincing and applicable classifiers for small datasets. In this paper, a minimum reconstruction error-based K-nearest neighbors (K-NN) classifier is proposed. We propose a new neighbor selection method. In the proposed neighbor selection method, a subset of data that minimize the reconstruction error is assigned as the neighbors of the query sample. Also, there is not any constraint on the distance of the neighbors from the query sample. An {{l}}_{0}-based sparse representation problem is introduced for selecting the neighbors. These neighbors are assigned as neighbors of the minimum reconstruction error-based K-NN classifiers. Three {{l}}_{0}-based minimum reconstruction error-based K-NN classifiers are introduced. These classifiers are less sensitive to the reconstruction coefficients in minimum reconstruction error-based K-NN classifiers and reconstruct the query sample with less error. The results on UCI machine learning repository, UCR time-series archive datasets, and a small subset (16%) of MNIST handwritten digit database demonstrate the suitable performance of the proposed method. The recognition precision increases up to more than 3% in some evaluations.

Exemplar-Based Sparse Representations for Detection of Parkinson's Disease From Speech

Parkinson's disease (PD) is a progressive neurological disorder which affects the motor system. The automatic detection of PD improves the diagnosis of the disease, and it can be done in a non-invasive manner from speech. In this paper, we investigate the use of an exemplar-based sparse representation (SR) classification approach for detecting PD from speech. Exemplars are speech feature vectors extracted from the training data. The idea is to formulate the detection task as a problem of finding sparse representations of test speech feature vectors with respect to training speech exemplars. The main advantage of using the SR approach instead of conventional machine learning (ML)-based approaches is that the training step–which is time-consuming and sometimes requires unorganized hyper-parameter tuning–is not needed. Furthermore, SRs are more robust to redundancy and noise in the data. In this work, we study SR classification approaches based on two sparse coding models, namely, l1-regularized least squares ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$l_{1}$</tex-math></inline-formula> LS) and non-negative least squares (NNLS). We propose a strategy based on class-specific dictionaries for improving performance of the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$l_{1}$</tex-math></inline-formula> LS- and NNLS-based SR classification. To investigate the detection performance, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$l_{1}$</tex-math></inline-formula> LS- and NNLS-based approaches are applied and compared with the traditional PD detection approach based on ML classification algorithms using the PC-GITA PD dataset and an openly available dataset consisting of mobile device voice recordings from healthy and PD patients. The results indicate that the proposed NNLS-based SR classification approach performs better than the traditional ML-based methods in discriminating PD patients from healthy subjects.

Parametric Instantaneous Frequency Estimation via PWSR With Adaptive QFM Dictionary

In this letter, an effective parametric IF estimation method based on piece-wise sparse representation (PWSR) is proposed to enhance the precision of conventional time-frequency distribution (TFD)-based estimators. Firstly, the signal is divided into a series of short-time segments according to the piece-wise quadratic frequency modulation (QFM) model. Then, the corresponding QFM parameters of each segment are accurately estimated by solving a sparse representation (SR) problem. Moreover, a construction scheme enabling the QFM dictionary to vary adaptively with the analyzed short-time signal is also provided. Finally, the IF of each segment is reconstructed according to the SR solution individually and combined together to generate the complete IF estimates. A radar imaging example verifies that the proposed method achieves a notable improvement in estimation accuracy as compared to existing TFD-based approaches.

A Target Recognition Method Based on Multiview Infrared Images

Infrared image target recognition provides an important means of night traffic management and battlefield environment monitoring. With the improvement of the performance of infrared sensors and the popularization of applications, it becomes possible to obtain multiview infrared images of the same target in the same scene. A target recognition method combining multiview infrared images is proposed. At first, the internal correlation analysis of multiview infrared images is performed based on the nonlinear correlation information entropy (NCIE). The view subset from all the multiview images with the largest NCIE is selected as candidate samples for the subsequent target recognition. The joint sparse representation (JSR) is used to classify all infrared images in the candidate view subset. JSR can effectively investigate the internal correlation of multiple related sparse representation problems and improve the reconstruction accuracy and classification capabilities. In the experiments, the tests are performed on the collected infrared images of multiple types of traffic vehicles, under the conditions of original, noisy, and occluded samples. The effectiveness and robustness of the proposed method can be verified by comparative analysis.

Feature Extraction of Weak-Bearing Faults Based on Laplace Wavelet and Orthogonal Matching Pursuit

The transient impact component of early bearing faults is not obvious, and the traditional basis function expansion method is poor in feature extraction under strong noise conditions. In this paper, a transient feature extraction technique is proposed based on Laplace wavelet and orthogonal matching pursuit algorithm and combined with sparse representation theory. First, the overcomplete and redundant Laplace wavelet dictionary is adopted to represent vibration signals in a sparse way. Then the Hilbert resonance demodulation method is employed to obtain the envelope spectrum of sparse representation signal. Finally, the coefficient calculation problem of sparse representation is solved by orthogonal matching pursuit (OMP) algorithm. Simulation examples and experimental example are used to examine the performance of the proposed method. The results show that the weak-bearing faults feature extraction can be effectively realized since the transient shock component can be identified. Furthermore, the effectiveness of the proposed method is verified by the cyclic multishock simulated signals as well as the practical rolling-bearing vibration signals. Moreover, the comparison studies are also carried out to show that the proposed method outperforms the traditional basis function expansion methods in weak fault feature extraction of bearing.

HP‐SPC algorithm with dynamic partially connected structure for mmWave MIMO systems

AbstractThis article proposes a low complexity hybrid precoding algorithm for the switch network‐based dynamic partially connected (SPC) structure, that is, named as HP‐SPC algorithm. First, via a new defined effective optimal precoding matrix, the analog switch precoding matrix optimization problem is transformed into a sparse representation problem. Thus, invoking that the key characteristic of only one nonzero entry in each row of the analog dynamic switch precoding matrix, the analog dynamic switch precoding matrix can be accurately and effectively solved. Second, the digital precoding matrix optimization problem is modeled as a dictionary update problem by the defined effective optimal precoding matrix and the combining matrix. Further, the digital precoding matrix is easily optimized based on the defined effective optimal precoding matrix, since the measurement vectors are sparsely represented by a single dictionary atom. Third, the analog phase shifter precoding matrix is given by the phase rotation method. Finally, the precoding matrices are alternant updated until convergence, a near optimal solution of the precoding matrix is obtained. Compared with the previous works, the proposed HP‐SPC algorithm provides better hybrid precoding performance, for examples: (1) it avoids complexity computation such as matrix inversion and singular value decomposition, so that it presents a low computational complexity; (2) it has a favorable property of convergence since a stable point can be reached with about 10 loop iterations. Based on the simulation results, the effectiveness of HP‐SPC algorithm is further demonstrated.

Unit-norm tight frame-based sparse representation with application to speech inpainting

This paper deals with the unit-norm tight frames (UNTFs) that better represent and solve sparse representation problems. Under this sparse representation framework, a novel model termed tight dictionary learning (TDL) is proposed. Unlike those dictionaries that only focus on the signals of interest, TDL intends to learn a more comprehensive dictionary that better represents the signals with the UNTF properties to facilitate sparse recovery. To better describe the physical meanings, the normalization constraints on the dictionary columns are imposed. A gradient-based approach is developed to obtain the tight dictionary where the normalization constraints are embedded into the cost function by a parametrization method. The learned tight dictionary is applied to the speech inpainting problem to restore speech signals corrupted by data missing. In order to compensate the degradation effect of the data missing, a UNTF-based preconditioning technique is developed to reform the frame properties of the degraded system. Parametrization and gradient-based approaches similar to those applied in the TDL are adopted to design a tight preconditioner. Extensive experiments on speech denoising and speech segment restoration are conducted to demonstrate the superiority of the proposed schemes.

Two-Dimensional ISAR Fusion Imaging of Block Structure Targets

The range resolution and azimuth resolution are restricted by the limited transmitting bandwidth and observation angle in a monostatic radar system. To improve the two-dimensional resolution of inverse synthetic aperture radar (ISAR) imaging, a fast linearized Bregman iteration for unconstrained block sparsity (FLBIUB) algorithm is proposed to achieve multiradar ISAR fusion imaging of block structure targets. First, the ISAR imaging echo data of block structure targets is established based on the geometrical theory of the diffraction model. The multiradar ISAR fusion imaging is transformed into a signal sparse representation problem by vectorization operation. Then, considering the block sparsity of the echo data of block structure targets, the FLBIUB algorithm is utilized to achieve the block sparse signal reconstruction and obtain the fusion image. The algorithm further accelerates the iterative convergence speed and improves the imaging efficiency by combining the weighted back-adding residual and condition number optimization of the basis matrix. Finally, simulation experiments show that the proposed method can effectively achieve block sparse signal reconstruction and two-dimensional multiradar ISAR fusion imaging of block structure targets.

Smooth non-negative sparse representation for face and handwritten recognition

In sparse representation problem, there is always interest to reduce the solution space by introducing additional constraints. This can lead to efficient application-specific algorithms. Despite known advantages of sparsity and non-negativity for image data representation, limited studies have addressed these characteristics simultaneously, due to the challenges involved. In this paper, we propose a novel inexpensive sparse non-negative reconstruction method. We utilise a non-negativity penalty term within a convex function while imposing sparsity at the same time. Our method, termed as SnSA (smooth non-negative sparse approximation) applies a novel thresholding strategy on the sparse coefficients during the minimisation of the proposed convex function. The main advantage of SnSA algorithm is that hard zeroing the negative samples which leads to unstable and non-optimal sparse solution is avoided. Instead, a differentiable smoothing function is proposed that allows gradual suppression of negative samples leading to a sparse non-negative solution. This way, the algorithm is driven towards a solution with a balance in maximising the sparsity and minimising the reconstruction error. Our numerical and experimental results on both synthetic signals and well-established face and handwritten image databases, indicate higher classification performance of the proposed method compared to the state-of-the-art techniques.

Face recognition via weighted non-negative sparse representation

Face recognition is one of the most important tools of identification in biometrics. Face recognition has attracted great attention in the last decades and numerous algorithms have been proposed. Different researches have shown that face recognition with Sparse Representation based Classification (SRC) has great classification performance. In some applications such as face recognition, it is appropriate to limit the search space of sparse solver because of local minima problem. In this paper, we apply this limitation via two methods. In the first, we apply the nonnegative constraint of sparse coefficients. As finding the sparse representation is a problem with very local minima, at first we use a simple classifier such as nearest subspace and then add the obtained information of this classifier to the sparse representation problem with some weights. Based on this view, we propose Weighted Non-negative Sparse Representation WNNSR for the face recognition problem. A quick and effective way to identify faces based on the sparse representation (SR) is smoothed $L_0$-norm $(SL_0)$ approach. In this paper, we solve the WNNSR problem based on the $SL_0$ idea. This approach is called Weighted Non-Negative Smoothed $L_0$ norm $(WNNSL_0)$. The simulation results on the Extended Yale B database demonstrate that the proposed method has high accuracy in face recognition better than the ultramodern sparse solvers approach.

Two-dimensional multiradar ISAR fusion imaging based on fast linearized Bregman iteration algorithm

The two-dimensional (2D) resolution is poor due to the narrow transmitting bandwidth and the limited observation angle in monostatic ISAR imaging. A multiradar fusion imaging method based on fast linearized Bregman iteration (FLBI) algorithm is proposed to improve the 2D resolution of the ISAR imaging. First, the sparsity of the ISAR imaging echo data is exploited to establish the multiradar fusion ISAR imaging model based on sparse representation, which can be converted into a one-dimensional sparse vector reconstruction problem. Then, a sparse reconstruction method based on FLBI is proposed to solve the sparse representation problem with large scales and achieve the ISAR fusion imaging. Combined with the weighted back-adding residual and condition number optimization of the sensing matrix, the FLBI algorithm can further accelerate the iterative convergence speed. The proposed algorithm only involves matrix–vector multiplications and componentwise shrinkages, which greatly improves the imaging efficiency. Finally, the simulation results show that the proposed method can effectively improve the iterative convergence speed and achieve the better 2D ISAR fusion imaging.

Adaptive Sparse Cyclic Coordinate Descent for Sparse Frequency Estimation

The frequency estimation of multiple complex sinusoids in the presence of noise is important for many signal processing applications. As already discussed in the literature, this problem can be reformulated as a sparse representation problem. In this letter, such a formulation is derived and an algorithm based on sparse cyclic coordinate descent (SCCD) for estimating the frequency parameters is proposed. The algorithm adaptively reduces the size of the used frequency grid, which eases the computational burden. Simulation results revealed that the proposed algorithm achieves similar performance to the original formulation and the Root-multiple signal classification (MUSIC) algorithm in terms of the mean square error (MSE), with significantly less complexity.

Self-attention based sentiment analysis with effective embedding techniques

The problem of sparse vector representation exists in handling the large-scale data and also semantics of words are not considered in many existing works of sentiment analysis. Effective word embed...

Self-attention based sentiment analysis with effective embedding techniques

The problem of sparse vector representation exists in handling the large-scale data and also semantics of words are not considered in many existing works of sentiment analysis. Effective word embedding techniques improve the task of sentiment analysis by overcoming the above two problems. It is a challenging task, when the review is expressed in multiple sentences and the entire sentence needs to be considered instead of individual words to determine the sentiment. To achieve this task, we have proposed a novel LSTM-based deep learning architecture that applies a sentence embedding using universal sentence encoder along with an attention layer. To evaluate the proposed approach, we have carried out various experiments on IMDB data set. From the experimental results, it is observed that the proposed method is significantly better than the other word-embedding-based approaches with an improvement of 5% and we have achieved a F1 score of 89.12% for our approach.

Real-Valued Sparse Bayesian Learning for DOA Estimation With Arbitrary Linear Arrays

Sparse Bayesian learning (SBL) has become a popular approach for direction-of-arrival (DOA) estimation, but its computational complexity for Bayesian inference is quite high because calculating inverse of a large complex matrix per iteration is required. It is known that the computational load can be reduced by transforming the complex-valued problem into a real-valued one. However, the commonly used real-valued transformation works for uniform linear arrays (ULAs) only. In this paper, we propose a new real-valued transformation for DOA estimation with arbitrary linear arrays by exploiting the virtual steering of linear arrays. Then, we introduce an alternating optimization algorithm based on the variational Bayesian inference (VBI) methodology to iteratively obtain a stationary solution to the real-valued sparse representation problem. Because of utilizing the additional real-valued structure, the VBI scheme can achieve a better performance in terms of both estimation accuracy and computational complexity. Moreover, we embed the generalized approximate message passing (GAMP) into the VBI-based method for further complexity reduction. Although there may be a performance loss for the GAMP variant, simulation results reveal its substantial performance improvement over existing methods.

Stochastic Recursive Gradient Support Pursuit and Its Sparse Representation Applications

In recent years, a series of matching pursuit and hard thresholding algorithms have been proposed to solve the sparse representation problem with -norm constraint. In addition, some stochastic hard thresholding methods were also proposed, such as stochastic gradient hard thresholding (SG-HT) and stochastic variance reduced gradient hard thresholding (SVRGHT). However, each iteration of all the algorithms requires one hard thresholding operation, which leads to a high per-iteration complexity and slow convergence, especially for high-dimensional problems. To address this issue, we propose a new stochastic recursive gradient support pursuit (SRGSP) algorithm, in which only one hard thresholding operation is required in each outer-iteration. Thus, SRGSP has a significantly lower computational complexity than existing methods such as SG-HT and SVRGHT. Moreover, we also provide the convergence analysis of SRGSP, which shows that SRGSP attains a linear convergence rate. Our experimental results on large-scale synthetic and real-world datasets verify that SRGSP outperforms state-of-the-art related methods for tackling various sparse representation problems. Moreover, we conduct many experiments on two real-world sparse representation applications such as image denoising and face recognition, and all the results also validate that our SRGSP algorithm obtains much better performance than other sparse representation learning optimization methods in terms of PSNR and recognition rates.