Standard Nonnegative Matrix Factorization Research Articles

Alternating Direction Method of Multipliers for Convolutive Non-Negative Matrix Factorization.

Non-negative matrix factorization (NMF) has become a popular method for learning interpretable patterns from data. As one of the variants of standard NMF, convolutive NMF (CNMF) incorporates an extra time dimension to each basis, known as convolutive bases, which is well suited for representing sequential patterns. Previously proposed algorithms for solving CNMF use multiplicative updates which can be derived by either heuristic or majorization-minimization (MM) methods. However, these algorithms suffer from problems, such as low convergence rates, difficulty to reach exact zeroes during iterations and prone to poor local optima. Inspired by the success of alternating direction method of multipliers (ADMMs) on solving NMF, we explore variable splitting (i.e., the core idea of ADMM) for CNMF in this article. New closed-form algorithms of CNMF are derived with the commonly used β -divergences as optimization objectives. Experimental results have demonstrated the efficacy of the proposed algorithms on their faster convergence, better optima, and sparser results than state-of-the-art baselines.

Hypergraph-Regularized Lp Smooth Nonnegative Matrix Factorization for Data Representation

Nonnegative matrix factorization (NMF) has been shown to be a strong data representation technique, with applications in text mining, pattern recognition, image processing, clustering and other fields. In this paper, we propose a hypergraph-regularized Lp smooth nonnegative matrix factorization (HGSNMF) by incorporating the hypergraph regularization term and the Lp smoothing constraint term into the standard NMF model. The hypergraph regularization term can capture the intrinsic geometry structure of high dimension space data more comprehensively than simple graphs, and the Lp smoothing constraint term may yield a smooth and more accurate solution to the optimization problem. The updating rules are given using multiplicative update techniques, and the convergence of the proposed method is theoretically investigated. The experimental results on five different data sets show that the proposed method has a better clustering effect than the related state-of-the-art methods in the vast majority of cases.

Robustness of NMF Algorithms Under Different Noises

In machine learning, datasets are often disturbed by different noises. The Nonnegative Matrix Factorization (NMF) algorithm provides a robust method to deal with noise, which will significantly improve the efficiency of machine learning. In this investigation, the standard NMF algorithm and L2,1-Norm Based NMF algorithm are studied by designing experiments on different noise types, noise levels, and datasets. Furthermore, Relative Reconstruction Errors (RRE), accuracy, and Normalized Mutual Information (NMI) are used to evaluate the robustness of the two algorithms. In this experiment, there is no significant difference in performance between the two algorithms, while L2,1-Norm Based NMF algorithm shows relatively small advantages.

Multiple Graph Adaptive Regularized Semi-Supervised Nonnegative Matrix Factorization with Sparse Constraint for Data Representation

Multiple graph and semi-supervision techniques have been successfully introduced into the nonnegative matrix factorization (NMF) model for taking full advantage of the manifold structure and priori information of data to capture excellent low-dimensional data representation. However, the existing methods do not consider the sparse constraint, which can enhance the local learning ability and improve the performance in practical applications. To overcome this limitation, a novel NMF-based data representation method, namely, the multiple graph adaptive regularized semi-supervised nonnegative matrix factorization with sparse constraint (MSNMFSC) is developed in this paper for obtaining the sparse and discriminative data representation and increasing the quality of decomposition of NMF. Particularly, based on the standard NMF, the proposed MSNMFSC method combines the multiple graph adaptive regularization, the limited supervised information and the sparse constraint together to learn the more discriminative parts-based data representation. Moreover, the convergence analysis of MSNMFSC is studied. Experiments are conducted on several practical image datasets in clustering tasks, and the clustering results have shown that MSNMFSC achieves better performance than several most related NMF-based methods.

An NMF-based method for jointly handling mixture nonlinearity and intraclass variability in hyperspectral blind source separation

Considering a set of observed signals that result from mixing (i.e. combining) a set of unknown source signals by means of an unknown function, blind source separation (BSS) and blind mixture identification (BMI) methods aim at estimating these source signals and/or the mixing function. Such methods have been extensively used, e.g. to process hyperspectral data in the field of remote sensing (i.e. Earth observation). However, most of these methods are restricted to the simplest mixing model, namely linear mixtures without the “variability” defined below. Some investigations started to tackle the more complex case of linear-quadratic (LQ), i.e. second-order polynomial, and memoryless mixtures. Some other investigations deal with the so-called intraclass variability phenomenon, i.e. with the case when each source yields a component having a somewhat different “shape” in each observed signal. Such nonlinearity and variability may e.g. appear in remote sensing hyperspectral images, as detailed in this paper. To our knowledge, each BSS/BMI method from the literature related to the nonlinearity and variability issues only addresses one of them. In contrast, we here propose a more powerful, very generic, BSS/BMI approach, that extends standard Nonnegative Matrix Factorization so as to jointly handle LQ mixtures and arbitrary source intraclass variability. This opens the way to a much wider range of BSS/BMI applications than plain linear “variabilityless” mixtures, e.g. to handle the above-defined two issues of remote sensing. Considering the latter application, we built data sets so that they are both realistic and suited to a detailed quantitative performance evaluation. The tests reported in this paper show that our method significantly outperforms several approaches from the literature: the estimation errors are thus decreased by factors up to around 2 and 1.5, respectively for spectra and mixing coefficients.

Diamond nitrogen-vacancy center charge state ratio determination at a given sample point

A new, all-optical, relatively simple, and fast method for determining the concentration ratio of negatively charged (NV−) to neutral (NV0) nitrogen-vacancy centers at a given microdiamond point is proposed. The photoluminescence spectra of NV centers were measured at various excitation powers exploiting the photoinduced interconversion effect. An isoemission point was found, and a range of incident laser intensity was determined, under which the total concentration of NV centers of both charge states was conserved, justifying decomposition of the set of photoluminescence curves into NV− and NV0 emission spectra by the standard non-negative matrix factorization (NNMF) procedure. Based on these results, the relative photoluminescence rate of NV− to NV0 centers for HPHT microdiamond was obtained with high accuracy. Possible applications of the new method are discussed.

Nonnegative matrix factorization with entropy regularization for hyperspectral unmixing

ABSTRACT Nonnegative matrix factorization (NMF) has been one of the most widely used techniques for hyperspectral unmixing (HU), which aims at decomposing each mixed pixel into a set of endmembers and their corresponding fractional abundances. However, the standard NMF model is ill-posed with only considering the non-negativity constraint. Therefore, many kinds of regularization (e.g. Tikhonov or sparsity regularization) have been imposed into NMF to well-define the model. Different from the general regularization, we introduce the entropy regularization into the NMF and propose an entropy regularized NMF (ERNMF) model for HU. In ERNMF, we minimize the entropy of that abundances on each pixel, which can achieve the sparsity of abundances. We also introduce a strategy to adaptively adjust the regularization parameter. In addition, we explore the proposed ERNMF with two optimization algorithms and provide the corresponding convergence and complexity analysis. Experimental results on both simulated and real-world data sets demonstrate the effectiveness of our proposed model and algorithms in comparison to the state-of-the-art approaches.

Generalized Separable Nonnegative Matrix Factorization.

Nonnegative matrix factorization (NMF) is a linear dimensionality technique for nonnegative data with applications such as image analysis, text mining, audio source separation, and hyperspectral unmixing. Given a data matrix M and a factorization rank r, NMF looks for a nonnegative matrix W with r columns and a nonnegative matrix H with r rows such that M ≈ WH. NMF is NP-hard to solve in general. However, it can be computed efficiently under the separability assumption which requires that the basis vectors appear as data points, that is, that there exists an index set K such that W = M(:,K). In this article, we generalize the separability assumption. We only require that for each rank-one factor W(:,k)H(k,:) for k=1,2,…,r, either W(:,k) = M(:,j) for some j or H(k,:) = M(i,:) for some i. We refer to the corresponding problem as generalized separable NMF (GS-NMF). We discuss some properties of GS-NMF and propose a convex optimization model which we solve using a fast gradient method. We also propose a heuristic algorithm inspired by the successive projection algorithm. To verify the effectiveness of our methods, we compare them with several state-of-the-art separable NMF and standard NMF algorithms on synthetic, document and image data sets.

Cluster-Wise Weighted NMF for Hyperspectral Images Unmixing with Imbalanced Data

Hyperspectral unmixing is an important technique for analyzing remote sensing images which aims to obtain a collection of endmembers and their corresponding abundances. In recent years, non-negative matrix factorization (NMF) has received extensive attention due to its good adaptability for mixed data with different degrees. The majority of existing NMF-based unmixing methods are developed by incorporating additional constraints into the standard NMF based on the spectral and spatial information of hyperspectral images. However, they neglect to exploit the nature of imbalanced pixels included in the data, which may cause the pixels mixed with imbalanced endmembers to be ignored, and thus the imbalanced endmembers generally cannot be accurately estimated due to the statistical property of NMF. To exploit the information of imbalanced samples in hyperspectral data during the unmixing procedure, in this paper, a cluster-wise weighted NMF (CW-NMF) method for the unmixing of hyperspectral images with imbalanced data is proposed. Specifically, based on the result of clustering conducted on the hyperspectral image, we construct a weight matrix and introduce it into the model of standard NMF. The proposed weight matrix can provide an appropriate weight value to the reconstruction error between each original pixel and the reconstructed pixel in the unmixing procedure. In this way, the adverse effect of imbalanced samples on the statistical accuracy of NMF is expected to be reduced by assigning larger weight values to the pixels concerning imbalanced endmembers and giving smaller weight values to the pixels mixed by majority endmembers. Besides, we extend the proposed CW-NMF by introducing the sparsity constraints of abundance and graph-based regularization, respectively. The experimental results on both synthetic and real hyperspectral data have been reported, and the effectiveness of our proposed methods has been demonstrated by comparing them with several state-of-the-art methods.

Ray-Space-Based Multichannel Nonnegative Matrix Factorization for Audio Source Separation

Nonnegative matrix factorization (NMF) has been traditionally considered a promising approach for audio source separation. While standard NMF is only suited for single-channel mixtures, extensions to consider multi-channel data have been also proposed. Among the most popular alternatives, multichannel NMF (MNMF) and further derivations based on constrained spatial covariance models have been successfully employed to separate multi-microphone convolutive mixtures. This letter proposes a MNMF extension by considering a mixture model with Ray-Space-transformed signals, where magnitude data successfully encodes source locations as frequency-independent linear patterns. We show that the MNMF algorithm can be seamlessly adapted to consider Ray-Space-transformed data, providing competitive results with recent state-of-the-art MNMF algorithms in a number of configurations using real recordings.

Self-Paced Nonnegative Matrix Factorization for Hyperspectral Unmixing

The presence of mixed pixels in the hyperspectral data makes unmixing to be a key step for many applications. Unsupervised unmixing needs to estimate the number of endmembers, their spectral signatures, and their abundances at each pixel. Since both endmember and abundance matrices are unknown, unsupervised unmixing can be considered as a blind source separation problem and can be solved by nonnegative matrix factorization (NMF). However, most of the existing NMF unmixing methods use a least-squares objective function that is sensitive to the noise and outliers. To deal with different types of noises in hyperspectral data, such as the noise in different bands (band noise), the noise in different pixels (pixel noise), and the noise in different elements of hyperspectral data matrix (element noise), we propose three self-paced learning based NMF (SpNMF) unmixing models in this article. The SpNMF models replace the least-squares loss in the standard NMF model with weighted least-squares losses and adopt a self-paced learning (SPL) strategy to learn the weights adaptively. In each iteration of SPL, atoms (bands or pixels or elements) with weight zero are considered as complex atoms and are excluded, while atoms with nonzero weights are considered as easy atoms and are included in the current unmixing model. By gradually enlarging the size of the current model set, SpNMF can select atoms from easy to complex. Usually, noisy or outlying atoms are complex atoms that are excluded from the unmixing model. Thus, SpNMF models are robust to noise and outliers. Experimental results on the simulated and two real hyperspectral data sets demonstrate that our proposed SpNMF methods are more accurate and robust than the existing NMF methods, especially in the case of heavy noise.

Generalized Morphological Component Analysis for Hyperspectral Unmixing

Hyperspectral unmixing (HU) is an active research topic in the remote-sensing community. It aims at modeling mixed pixels using a collection of pure constituent materials ( endmembers ) weighted by their corresponding fractional abundances. Among existing unmixing schemes, nonnegative matrix factorization (NMF) has drawn significant attention due to its unsupervised nature, as well as its capacity to obtain both endmembers and fractional abundances simultaneously. In this article, we present a new blind unmixing method based on the generalized morphological component analysis (GMCA) framework, in which an additional constraint is introduced into the standard NMF model to represent the sparsity and morphological diversity of the abundance maps associated with each endmember. More specifically, we take into account the fact that different ground categories in a hyperspectral scene generally exhibit various spatial distributions and morphological characteristics. As a result, when providing a specific dictionary basis for these categories, their corresponding abundance maps (referred to as sources ) can be sparsely represented. In addition, due to the low correlation between different sources, their sparse representations will not share the same most significant coefficients. With this observation in mind, we can further promote source discrimination and separation in the unmixing process. Moreover, in order to obtain a stable solution of the involved optimization problem, we adopt an alternate iterative constrained algorithm with a threshold descent strategy. Our experiments, carried out on both synthetic and real hyperspectral scenes, reveal that our newly developed GMCA-based unmixing method obtains very promising results with fast convergence speed and requiring significantly less parameter tuning. This confirms the advantage of the proposed spatial morphological component approach for HU purposes.

A framework for least squares nonnegative matrix factorizations with Tikhonov regularization

Nonnegative matrix factorization (NMF) is widely used for dimensionality reduction, clustering and signal unmixing. This paper presents a generic model for least squares NMFs with Tikhonov regularization, which covers many well-known NMF models as well as new models. We also develop a generic updating rule with a simple structure to iteratively solve the optimization problem by constructing a surrogate function, which possesses properties similar to that of the standard NMF. The simulation results demonstrate the power of the framework in which some new algorithms can be derived to provide a performance superior to that of other commonly used methods.

Robust Bi-Stochastic Graph Regularized Matrix Factorization for Data Clustering.

Data clustering, which is to partition the given data into different groups, has attracted much attention. Recently various effective algorithms have been developed to tackle the task. Among these methods, non-negative matrix factorization (NMF) has been demonstrated to be a powerful tool. However, there are still some problems. First, the standard NMF is sensitive to noises and outliers. Although l2,1 norm based NMF improves the robustness, it is still affected easily by large noises. Second, for most graph regularized NMF, the performance highly depends on the initial similarity graph. Third, many graph-based NMF models perform the graph construction and matrix factorization in two separated steps. Thus the learned graph structure may not be optimal. To overcome the above drawbacks, we propose a robust bi-stochastic graph regularized matrix factorization (RBSMF) framework for data clustering. Specifically, we present a general loss function, which is more robust than the commonly used L2 and L1 functions. Besides, instead of keeping the graph fixed, we learn an adaptive similarity graph. Furthermore, the graph updating and matrix factorization are processed simultaneously, which can make the learned graph more appropriate for clustering. Extensive experiments have shown the proposed RBSMF outperforms other state-of-the-art methods.

Transductive Nonnegative Matrix Tri-Factorization

Nonnegative matrix factorization (NMF) decomposes a nonnegative matrix into the product of two lower-rank nonnegative matrices. Since NMF learns parts-based representation, it has been widely used as a feature learning component in many fields. However, standard NMF algorithms ignore the training labels as well as unlabeled data in the test domain. In this paper, we propose a transductive nonnegative matrix tri-factorization method (T-NMTF) to simultaneously exploit the label information of training examples and the statistical structure of features in the test domain. Different from standard NMF, nonnegative matrix tri-factorization (NMTF) decomposes a nonnegative matrix into the product of three lower-rank nonnegative matrices, and thus provides a flexible framework to transduce discriminative information of training examples to test examples. In particular, the proposed T-NMTF projects both training examples and test examples into a unified subspace, and expects the coefficients of training examples close to their label vectors. Since training examples and test examples are assumed to identically distributed, it is reasonable to expect the learned coefficients of test examples approximate their label vectors well. To estimate the T-NMTF parameters, we develop an efficient multiplicative update rule and prove its convergence. In addition, we propose a manifold regularized T-NMTF (MT-NMTF) algorithm that exploits the local geometry structure of the dataset to boost discriminant power. Experimental results on face recognition demonstrate the effectiveness of T-NMTF and MT-NMTF.

Hybrid projective nonnegative matrix factorization based on α-divergence and the alternating least squares algorithm

Nonnegative Matrix Factorization (NMF) is a linear dimensionality reduction technique for extracting hidden and intrinsic features of high-dimensional data sets. Recently, several Projective NMF (P-NMF) methods have been proposed for the purpose of resolving issues associated with the standard NMF approach. Experimental results show that P-NMF algorithms outperform the standard NMF method in some aspects. But some basic issues still affect the existing NMF and P-NMF methods, these include slow convergence rate, low reconstruction accuracy and dense basis factors. In this article, we propose a new and generalized hybrid algorithm by combining the concept of alternating least squares with the multiplicative update rules of the α-divergence-based P-NMF method. We have conducted extensive numerical experiments on 7 real-world data sets and compared the new algorithm with several state-of-the-art methods. The attractive features and added advantages of the new algorithm include remarkable clustering performances, providing highly “orthogonal” and very sparse basis factors, and extracting distinctive and better localized features of the original data than its counterparts.

Hypergraph Regularized Discriminative Nonnegative Matrix Factorization on Sample Classification and Co-Differentially Expressed Gene Selection

Nonnegative Matrix Factorization (NMF) is a significant big data analysis technique. However, standard NMF regularized by simple graph does not have discriminative function, and traditional graph models cannot accurately reflect the problem of multigeometry information between data. To solve the above problem, this paper proposed a new method called Hypergraph Regularized Discriminative Nonnegative Matrix Factorization (HDNMF), which captures intrinsic geometry by constructing hypergraphs rather than simple graphs. The introduction of the hypergraph method allows high-order relationships between samples to be considered, and the introduction of label information enables the method to have discriminative effect. Both the hypergraph Laplace and the discriminative label information are utilized together to learn the projection matrix in the standard method. In addition, we offered a corresponding multiplication update solution for the optimization. Experiments indicate that the method proposed is more effective by comparing with the earlier methods.

Blind Audio Source Separation With Minimum-Volume Beta-Divergence NMF

Considering a mixed signal composed of various audio sources and recorded with a single microphone, we consider on this paper the blind audio source separation problem which consists in isolating and extracting each of the sources. To perform this task, nonnegative matrix factorization (NMF) based on the Kullback-Leibler and Itakura-Saito $\beta$-divergences is a standard and state-of-the-art technique that uses the time-frequency representation of the signal. We present a new NMF model better suited for this task. It is based on the minimization of $\beta$-divergences along with a penalty term that promotes the columns of the dictionary matrix to have a small volume. Under some mild assumptions and in noiseless conditions, we prove that this model is provably able to identify the sources. In order to solve this problem, we propose multiplicative updates whose derivations are based on the standard majorization-minimization framework. We show on several numerical experiments that our new model is able to obtain more interpretable results than standard NMF models. Moreover, we show that it is able to recover the sources even when the number of sources present into the mixed signal is overestimated. In fact, our model automatically sets sources to zero in this situation, hence performs model order selection automatically.

Efficient algorithm for sparse symmetric nonnegative matrix factorization

Symmetric Nonnegative Matrix Factorization (symNMF) is a special case of the standard Nonnegative Matrix Factorization (NMF) method which is the most popular linear dimensionality reduction technique for analyzing nonnegative data. Examples of symmetric matrices that arise in real-life applications include covariance matrices in finance, adjacency matrices associated with undirected graphs, and distance matrices arising from video and other media summarization technologies. The advantages of having sparse factors in symNMF include saving a great deal of storage space and enhancing the extraction of localized basis features that better represent the latent features of the original data. In order to obtain sparser basis factors, we formulate a new sparse symNMF model by imposing an l1-norm based sparsity constraint on the symNMF problem. We use the concept of rank-one nonnegative matrix approximation to decouple this non-convex optimization problem into convex nonnegative least squares sub-problems which are easier to solve. We develop a new and efficient coordinate-descent based algorithm and test it on four well-known databases of facial images. The proposed algorithm is shown to have fast convergence, well-suited for solving the sparse symNMF problem, and extract localized basis features of image datasets that facilitate interpretation.

A novel framework for the NMF methods with experiments to unmixing signals and feature representation

Non-negative matrix factorization (NMF) can be used in clustering, feature representation or blind source separation. Many NMF methods have been developed including least squares (LS) error, Kullback–Leibler (KL) divergence, Itakura–Saito (IS) divergence, Bregman-divergence, α -divergence, β -divergence, γ -divergence, convex, constrained, graph-regularized NMFs. The main contribution of this paper is to develop a framework to generalize the existing NMF methods and also provide new NMF methods. This paper constructs a general optimization model and develops a generic updating rule with a simple structure using a surrogate function, which possesses similar properties as the standard NMF methods. The experimental results, obtained using several standard databases, demonstrate the power of the work in which some new methods provide performance superior to that of the other existing methods.