Nonnegative Representation Research Articles

Laplacian Regularized Nonnegative Representation for Clustering and Dimensionality Reduction

Self-representation methods, such as low-rank representation (LRR), sparse subspace clustering (SSC) and their variants may generate negative coding coefficients since there is no explicit nonnegative constraint. These negative coefficients lack physical meaning. To be specific, it is unreasonable to allow a query sample encoded by heterogeneous samples to cancel each other out with subtractions. In this paper, we propose a novel model named Laplacian regularized nonnegative representation (LapNR). The new model improves its physical interpretability by ensuring that the query sample should be approximated from homogeneous samples and irrelevant to heterogeneous ones. More importantly, it captures the geometric information of input data by imposing the graph Laplacian to the nonnegative representations. As a result, the representation matrix generated by our LapNR model becomes sparse and discriminative. Based on the alternating direction method of multipliers (ADMM), an efficient optimization procedure is developed for LapNR. The extensive experiments on clustering and dimensionality reduction tasks show the effectiveness and efficiency of our LapNR.

Graph Regularized Constrained Non-Negative Matrix Factorization With Lₚ Smoothness for Image Representation

Nonnegative matrix factorization-based image representation algorithms have been widely applied to deal with high-dimensional data in the past few years. In this paper, we propose a graph regularized constrained nonnegative matrix factorization with L p Smoothing (GCNMFS) for image representation. Specifically, the main contributions of the proposed GCNMFS method include as follows: firstly, the geometric manifold structure hidden in data is effectively exploited by adopting a graph regularizer. Secondly, the label information of labeled samples is incorporated into the model of NMF without additional parameters. Finally, the L p smoothness constraint is used to constrain the basis matrix, and thus a smooth and more accurate solution is produced. Moreover, an effective optimization scheme is presented to solve the proposed model. Extensive experiments on several image datasets show the proposed GCNMFS method can achieve better performance than other state-of-the-art methods in clustering.

Unsupervised Nonnegative Adaptive Feature Extraction for Data Representation

In this paper, we propose a novel unsupervised Nonnegative Adaptive Feature Extraction (NAFE) algorithm for data representation and classification. The formulation of NAFE integrates the sparsity constrained nonnegative matrix factorization (NMF), representation learning, and adaptive reconstruction weight learning into a unified model. Specifically, NAFE performs feature and weight learning over the new robust representations of NMF for more accurate measure and representation. For nonnegative adaptive feature extraction, our NAFE first utilizes the sparsity constrained NMF to obtain the new and robust representations of the original data. To preserve the manifold structures of the learnt new representations, we also incorporate a neighborhood reconstruction error over the weight matrix for joint minimization. Note that to further improve the representation power, the weights are jointly shared in the new low-dimensional nonnegative representation space, low-dimensional nonlinear manifold space, and low-dimensional projective subspace, i.e., local neighborhood information is clearly preserved in different feature spaces so that informative representations and features can be jointly obtained. To enable NAFE to extract features from new data, we also include a feature approximation error by a linear projection so that the learnt extractor can obtain features from new data efficiently. Extensive simulations show that our formulation can deliver state-of-the-art results on several public databases for feature extraction and classification, compared with several related methods.

Collaborative Representation Using Non-Negative Samples for Image Classification.

Collaborative representation based classification (CRC) is an efficient classifier in image classification. By using regularization, the collaborative representation based classifier holds competitive performances compared with the sparse representation based classifier using less computational time. However, each of the elements calculated from the training samples are utilized for representation without selection, which can lead to poor performances in some classification tasks. To resolve this issue, in this paper, we propose a novel collaborative representation by directly using non-negative representations to represent a test sample collaboratively, termed Non-negative Collaborative Representation-based Classifier (NCRC). To collect all non-negative collaborative representations, we introduce a Rectified Linear Unit (ReLU) function to perform filtering on the coefficients obtained by minimization according to CRC’s objective function. Next, we represent the test sample by using a linear combination of these representations. Lastly, the nearest subspace classifier is used to perform classification on the test samples. The experiments performed on four different databases including face and palmprint showed the promising results of the proposed method. Accuracy comparisons with other state-of-art sparse representation-based classifiers demonstrated the effectiveness of NCRC at image classification. In addition, the proposed NCRC consumes less computational time, further illustrating the efficiency of NCRC.

Switching Divergences for Spectral Learning in Blind Speech Dereverberation

When recorded in an enclosed room, a sound signal will most certainly get affected by reverberation. This not only undermines audio quality, but also poses a problem for many human-machine interaction technologies that use speech as their input. In this paper, a new blind, two-stage dereverberation approach based in a generalized $\beta$-divergence as a fidelity term over a non-negative representation is proposed. The first stage consists of learning the spectral structure of the signal solely from the observed spectrogram, while the second stage is devoted to model reverberation. Both steps are taken by minimizing a cost function in which the aim is put either in constructing a dictionary or a good representation by changing the divergence involved. In addition, an approach for finding an optimal fidelity parameter for dictionary learning is proposed. An algorithm for implementing the proposed method is described and tested against state-of-the-art methods. Results show improvements for both artificial reverberation and real recordings.

Sparse, collaborative, or nonnegative representation: Which helps pattern classification?

The use of sparse representation (SR) and collaborative representation (CR) for pattern classification has been widely studied in tasks such as face recognition and object categorization. Despite the success of SR/CR based classifiers, it is still arguable whether it is the ℓ1-norm sparsity or the ℓ2-norm collaborative property that brings the success of SR/CR based classification. In this paper, we investigate the use of nonnegative representation (NR) for pattern classification, which is largely ignored by previous work. Our analyses reveal that NR can boost the representation power of homogeneous samples while limiting the representation power of heterogeneous samples, making the representation sparse and discriminative simultaneously and thus providing a more effective solution to representation based classification than SR/CR. Our experiments demonstrate that the proposed NR based classifier (NRC) outperforms previous representation based classifiers. With deep features as inputs, it also achieves state-of-the-art performance on various visual classification tasks.

Safety Verification of Nonlinear Hybrid Systems Based on Bilinear Programming

In safety verification of hybrid systems, barrier certificates are generated by solving the verification conditions derived from non-negative representations of different types. This paper presents a new computational method, sequential linear programming projection, for directly solving the set of verification conditions represented by the Krivine–Vasilescu–Handelman’s positivstellensatz. The key idea is to decompose it into two successive optimization problems that refine the desired barrier certificate and those undetermined multipliers, respectively, and solve it in an iterative scheme. The most important benefit of the proposed approach lies in that it is much more effective than the LP relaxation method in producing real barrier certificates, and possesses a much lower computational complexity than the popular sum of square relaxation methods, which is demonstrated by the theoretical analysis on complexity and the experiment on a set of examples gathered from the literature.

Adaptive non-negative projective semi-supervised learning for inductive classification

We discuss the inductive classification problem by proposing a joint framework termed Adaptive Non-negative Projective Semi-Supervised Learning (ANP-SSL). Specifically, ANP-SSL integrates the adaptive inductive label propagation, adaptive reconstruction weights learning and the neighborhood preserving projective nonnegative matrix factorization (PNMF) explicitly. To make the label prediction results more accurate, ANP-SSL incorporates the semi-supervised data representation and classification errors into regular PNMF for minimization, which can enable our ANP-SSL to perform the adaptive weights learning and label propagation over the spatially local and part-based data representations, which differs from most existing work that usually assign weights and predict labels based on the original data that often has noise and corruptions. Moreover, existing methods usually pre-assign weights before the process of label estimation, but such operation cannot ensure the learnt weights by independent step to be optimal for the subsequent classification. The combined representation error can also make the learnt reduced part-based representations of neighborhood preserving PNMF, which can potentially enhance the prediction results. By minimizing the classification error jointly over the neighborhood preserving nonnegative representation can make the embedding based classification efficient. Extensive results on several public image databases verified the effectiveness of our ANP-SSL, compared with other state-of-the-art methods.

Joint Linear Regression and Nonnegative Matrix Factorization Based on Self-Organized Graph for Image Clustering and Classification

Nonnegative matrix factorization (NMF) technique has been developed successfully to represent the intuitively meaningful feature of data. A suitable representation can faithfully preserve the intrinsic structure of data. Due to the fact that it introduces the label information, semi-supervised NMF has been demonstrated more advantageous in image representation than original NMF. However, previous semi-supervised NMF variants construct a label indicator matrix only for tagging the labeled data and not being optimized together with the matrix factorization. It is short of label propagation and fails to work for predicting the attribution of data. Moreover, the transductive semi-supervised NMF variants cannot dispose the prediction of unseen data, restricting the application of NMF. In this paper, a joint optimization framework of linear regression and NMF (LR-NMF) based on the self-organized graph is proposed for a completed task which simultaneously takes into account image representation and attribution prediction. By minimizing the proposed objective, three interactive threads are running: decomposing the data into nonnegative basis matrix and the corresponding representation, linear regression using the nonnegative representation, and label propagation based on the self-organized graph which is defined in the feature space. The products of LR-NMF can be viewed as extracting nonnegative feature for clustering, meanwhile, they can be used to solve the out-of-sample problem for classification. Extensive clustering and classification experiments on the digit, face, and object challenging data sets are presented to show the efficacy of the proposed LR-NMF algorithm.

Co-regularized multiview nonnegative matrix factorization with correlation constraint for representation learning

With the increasing availability of multiview nonnegative data in real applications, multiview representation learning based on nonnegative matrix factorization (NMF) has attracted more and more attentions. However, existing NMF-based methods are sensitive to noises and are difficult to generate discriminative features with noisy views. To address these problems, we propose a co-regularized multiview nonnegative matrix factorization method with correlation constraint for nonnegative representation learning, which jointly exploits consistent and complementary information across different views. Different from previous works, we aim at integrating information from multiple views efficiently and making it more robust to the presence of noisy views. More specifically, we exploit the complementary information of multiple views through the co-regularization to accommodate the presence of the noisy views. Meanwhile, correlation constraint is imposed on the low-dimensional space to learn a common latent representation shared by different views. For the induced objective function, we derive an alternative algorithm to solve the optimization problem. The experimental results on four real datasets demonstrate the effectiveness and robustness of the proposed algorithm.

Robust Control of Varying Weak Hyperspectral Target Detection With Sparse Nonnegative Representation

In this study, a multiple-comparison approach is developed for detecting faint hyperspectral sources. The detection method relies on a sparse and non-negative representation on a highly coherent dictionary to track a spatially varying source. A robust control of the detection errors is ensured by learning the test statistic distributions on the data. The resulting control is based on the false discovery rate, to take into account the large number of pixels to be tested. This method is applied to data recently recorded by the three-dimensional spectrograph Multi-Unit Spectrograph Explorer.

Multi-View Clustering via Deep Matrix Factorization

Multi-View Clustering (MVC) has garnered more attention recently since many real-world data are comprised of different representations or views. The key is to explore complementary information to benefit the clustering problem. In this paper, we present a deep matrix factorization framework for MVC, where semi-nonnegative matrix factorization is adopted to learn the hierarchical semantics of multi-view data in a layer-wise fashion. To maximize the mutual information from each view, we enforce the non-negative representation of each view in the final layer to be the same. Furthermore, to respect the intrinsic geometric structure in each view data, graph regularizers are introduced to couple the output representation of deep structures. As a non-trivial contribution, we provide the solution based on alternating minimization strategy, followed by a theoretical proof of convergence. The superior experimental results on three face benchmarks show the effectiveness of the proposed deep matrix factorization model.

A low-rank matrix factorization approach for joint harmonic and baseline noise suppression in biopotential signals

Background and objectivesIn this paper we propose a novel single-channel harmonic and baseline noise removal approach based on the low-rank matrix factorization theory. It aims to enhance spectrogram sparsity in order to significantly reduce the dimensionality of the underlying sources in the input data. Such a low-rank non-negative representation approach admits efficient noise removal. MethodsThe sparsity is improved by a modification of the time-frequency basis through the following signal processing steps: (1) spectrograms segmentation, (2) non-negative rank estimation, and (3) source grouping. The source waveforms are retrieved by means of non-negative matrix factorization and the overlap-add method. The proposed method was tested on real electrocardiogram and electromyogram signals for different analysis scenarios, against two state-of-the-art reference methods. ResultsPerformance evaluation was carried out by means of the output signal-to-interference ratio. In the electrocardiogram analysis scenarios, for the input signal-to-interference ratio as low as −15dB, the proposed method outperforms the reference methods by 8dB and 17dB respectively. Regarding electromyogram denoising, the performance improvement is about 3dB. ConclusionsThe proposed method was shown to be very efficient in harmonic and baseline simultaneous removing from electrocardiogram and electromyogram signals. Its structure allows for a straightforward extension to other biopotential signals e.g. electroencephalograms and multichannel processing.

Semi-supervised Nonnegative Matrix Factorization with Commonness Extraction

Standard nonnegative matrix factorization extracts nonnegative bases for nonnegative representation, which, however, considers only features, not commonness. In addition, the standard NMF is an unsupervised learning method that cannot fully utilize label information if it exists. In this paper, we present a semi-supervised commonness NMF technique that incorporates samples' commonness and label information into the optimization model. Naturally, the commonness vector should be constrained by nonnegativity and will degenerate to zero if no commonness exists. We develop a multiplicative update rule to solve the model, which has properties comparable to those of the standard NMF with automatic satisfaction of the nonnegativity constraints, monotonicity without the need for any adjustable learning rate and a low computational overhead. Through experiments on the standard databases, we analyze the behavior of the proposed method, which exhibits a performance that is favorably superior with respect to commonness extraction and clustering accuracy.

A non-negative low-rank representation for hyperspectral band selection

ABSTRACTHyperspectral images are widely used in real applications due to their rich spectral information. However, the large volume brings a lot of inconvenience, such as storage and transmission. Hyperspectral band selection is an important technique to cope with this issue by selecting a few spectral bands to replace the original image. This article proposes a novel band selection algorithm that first estimates the redundancy through analysing relationships among spectral bands. After that, spectral bands are ranked according to their relative importance. Subsequently, in order to remove redundant spectral bands and preserve the original information, a maximal linearly independent subset is constructed as the optimal band combination. Contributions of this article are listed as follows: (1) A new strategy for band selection is proposed to preserve the original information mostly; (2) A non-negative low-rank representation algorithm is developed to discover intrinsic relationships among spectral bands; (3) A smart strategy is put forward to adaptively determine the optimal combination of spectral bands. To verify the effectiveness, experiments have been conducted on both hyperspectral unmixing and classification. For unmixing, the proposed algorithm decreases the average root mean square errors (RMSEs) by 0.05, 0.03, and 0.05 for the Urban, Cuprite, and Indian Pines data sets, respectively. With regard to classification, our algorithm achieves the overall accuracies of 77.07% and 89.19% for the Indian Pines and Pavia University data sets, respectively. These results are close to the performance with original images. Thus, comparative experiments not only illustrate the superiority of the proposed algorithm, but also prove the validity of band selection on hyperspectral image processing.

Bidirectional Nonnegative Deep Model and Its Optimization in Learning

Nonnegative matrix factorization (NMF) has been successfully applied in signal processing as a simple two-layer nonnegative neural network. Projective NMF (PNMF) with fewer parameters was proposed, which projects a high-dimensional nonnegative data onto a lower-dimensional nonnegative subspace. Although PNMF overcomes the problem of out-of-sample of NMF, it does not consider the nonlinear characteristic of data and is only a kind of narrow signal decomposition method. In this paper, we combine the PNMF with deep learning and nonlinear fitting to propose a bidirectional nonnegative deep learning (BNDL) model and its optimization learning algorithm, which can obtain nonlinear multilayer deep nonnegative feature representation. Experiments show that the proposed model can not only solve the problem of out-of-sample of NMF but also learn hierarchical nonnegative feature representations with better clustering performance than classical NMF, PNMF, and Deep Semi-NMF algorithms.

Optimized projections for nonnegative linear reconstruction classification

Nonnegative linear reconstruction measure can produce the sparse nonnegative representation coefficients which are concentrated on the similar training samples. The samples with the same class-label are generally more similar than those with different class-labels. Based on this, we develop a nonnegative linear reconstruction based classifiers (NLRC). To enhance the classification performance, we design a feature extractor based on the same decisional rule of NLRC, such that the extracted features are most suitable to NLRC. The proposed feature extractor, nonnegative linear reconstruction projection (NLRP), and NLRC share a common decisional rule, and thus the features extracted by NLRP fit NLRC well in theory. The feasibility and effectiveness of the proposed recognition system are verified on the Yale face database, the AR face database the PolyU finger knuckle print database, and the palmprint database with promising results.

Fast Non-Negative Orthogonal Matching Pursuit

One of the important classes of sparse signals is the non-negative signals. Many algorithms have already been proposed to recover such non-negative representations, where greedy and convex relaxed algorithms are among the most popular methods. The greedy techniques have been modified to incorporate the non-negativity of the representations. One such modification has been proposed for Orthogonal Matching Pursuit (OMP), which first chooses positive coefficients and uses a non-negative optimisation technique as a replacement for the orthogonal projection onto the selected support. Beside the extra computational costs of the optimisation program, it does not benefit from the fast implementation techniques of OMP. These fast implementations are based on the matrix factorisations. We here first investigate the problem of positive representation, using pursuit algorithms. We will then describe a new implementation, which can fully incorporate the positivity constraint of the coefficients, throughout the selection stage of the algorithm. As a result, we present a novel fast implementation of the Non-Negative OMP, which is based on the QR decomposition and an iterative coefficients update. We will empirically show that such a modification can easily accelerate the implementation by a factor of ten in a reasonable size problem.

A non-negative representation learning algorithm for selecting neighbors

A crucial step in many manifold clustering and embedding algorithms involves selecting the neighbors of a data point. In this paper, a non-negative representation learning algorithm is proposed to select the neighbors of a data point, so that they are almost always lying in the tangent space of the manifold at the given query point. This is very important to multi-manifold clustering and manifold embedding. Convex geometry theory states that a data point on a face of a convex polytope can be a convex combination of the vertices of the polytope, and that the non-zero combination scalars correspond to the vertices of the face the point lies on. The intent of this paper is to use points that that are near to a point as the vertices of a convex polytope, and select the vertices on the same face as this point to be the neighbors. Clearly, the point can be a convex combination of its neighbors. We can observe that these neighbors almost always lie in the tangent space of the manifold at the query point and can preserve the local manifold structure well. Based on this basic idea, we construct an objective function. Moreover, an augmented Lagrange multipliers method is proposed for solving it to derive a non-negative representation, and then the neighbors of a data point can be obtained. The result of our neighbor selection algorithm can be used by other clustering methods such as LEM or manifold embedding methods such as LLE. We demonstrate the effectiveness and efficiency of the proposed method through experiments on synthetic data and a real-world problem.

A Local Non-Negative Pursuit Method for Intrinsic Manifold Structure Preservation

The local neighborhood selection plays a crucial role for most representation based manifold learning algorithms. This paper reveals that an improper selection of neighborhood for learning representation will introduce negative components in the learnt representations. Importantly, the representations with negative components will affect the intrinsic manifold structure preservation. In this paper, a local non-negative pursuit (LNP) method is proposed for neighborhood selection and non-negative representations are learnt. Moreover, it is proved that the learnt representations are sparse and convex. Theoretical analysis and experimental results show that the proposed method achieves or outperforms the state-of-the-art results on various manifold learning problems.