Standard Nonnegative Matrix Factorization Algorithms Research Articles

Robustness of NMF Algorithms Under Different Noises

In machine learning, datasets are often disturbed by different noises. The Nonnegative Matrix Factorization (NMF) algorithm provides a robust method to deal with noise, which will significantly improve the efficiency of machine learning. In this investigation, the standard NMF algorithm and L2,1-Norm Based NMF algorithm are studied by designing experiments on different noise types, noise levels, and datasets. Furthermore, Relative Reconstruction Errors (RRE), accuracy, and Normalized Mutual Information (NMI) are used to evaluate the robustness of the two algorithms. In this experiment, there is no significant difference in performance between the two algorithms, while L2,1-Norm Based NMF algorithm shows relatively small advantages.

Generalized Separable Nonnegative Matrix Factorization.

Nonnegative matrix factorization (NMF) is a linear dimensionality technique for nonnegative data with applications such as image analysis, text mining, audio source separation, and hyperspectral unmixing. Given a data matrix M and a factorization rank r, NMF looks for a nonnegative matrix W with r columns and a nonnegative matrix H with r rows such that M ≈ WH. NMF is NP-hard to solve in general. However, it can be computed efficiently under the separability assumption which requires that the basis vectors appear as data points, that is, that there exists an index set K such that W = M(:,K). In this article, we generalize the separability assumption. We only require that for each rank-one factor W(:,k)H(k,:) for k=1,2,…,r, either W(:,k) = M(:,j) for some j or H(k,:) = M(i,:) for some i. We refer to the corresponding problem as generalized separable NMF (GS-NMF). We discuss some properties of GS-NMF and propose a convex optimization model which we solve using a fast gradient method. We also propose a heuristic algorithm inspired by the successive projection algorithm. To verify the effectiveness of our methods, we compare them with several state-of-the-art separable NMF and standard NMF algorithms on synthetic, document and image data sets.

Transductive Nonnegative Matrix Tri-Factorization

Nonnegative matrix factorization (NMF) decomposes a nonnegative matrix into the product of two lower-rank nonnegative matrices. Since NMF learns parts-based representation, it has been widely used as a feature learning component in many fields. However, standard NMF algorithms ignore the training labels as well as unlabeled data in the test domain. In this paper, we propose a transductive nonnegative matrix tri-factorization method (T-NMTF) to simultaneously exploit the label information of training examples and the statistical structure of features in the test domain. Different from standard NMF, nonnegative matrix tri-factorization (NMTF) decomposes a nonnegative matrix into the product of three lower-rank nonnegative matrices, and thus provides a flexible framework to transduce discriminative information of training examples to test examples. In particular, the proposed T-NMTF projects both training examples and test examples into a unified subspace, and expects the coefficients of training examples close to their label vectors. Since training examples and test examples are assumed to identically distributed, it is reasonable to expect the learned coefficients of test examples approximate their label vectors well. To estimate the T-NMTF parameters, we develop an efficient multiplicative update rule and prove its convergence. In addition, we propose a manifold regularized T-NMTF (MT-NMTF) algorithm that exploits the local geometry structure of the dataset to boost discriminant power. Experimental results on face recognition demonstrate the effectiveness of T-NMTF and MT-NMTF.

Extraction of Muscle Synergies in Spinal Cord Injured Patients.

Muscle synergies encode motor activity as a linear superposition of multiple motor units composed of a temporal command exciting a specific network of muscles. This study examines muscle synergies derived from simple standing studies of a complete spinal cord injury (SCI) patient under epidural spinal stimulation. A popular technique for extracting these synergies from EMG data is non-negative matrix factorization (NNMF). However, standard NNMF algorithms do not allow for physiological delays for a neural signal to reach different muscles. These delays are prevalent in SCI patients under spinal stimulation, and so we propose a new algorithm (regularized ShiftNMF) to extract muscle synergies which account for signal delays. We find muscle synergies extracted by the regularized ShiftNMF algorithm are significantly better at reconstructing EMG activity, and the resulting features are physiologically consistent and more useful in describing patient behavior.

Manifold NMF with L21 norm for clustering

Nonnegative matrix factorization has been widely used in data mining and machine learning fields as a clustering algorithm. The standard nonnegative matrix factorization algorithm utilizes the sum of squared error to measure the quality of factorization, however, the noise and outliers in the dataset will reduce the performance of algorithm significantly. This paper proposes a robust manifold nonnegative matrix factorization algorithm based on L21 norm, and the projected gradient method is utilized to obtain the updating rules. The proposed algorithm utilizes the L21 norm to measure the quality of factorization, which is insensitive to the noise and outliers, also it utilizes the geometrical structure of the dataset and considers the local invariance. The experimental results on several data sets and the comparison with other clustering algorithms demonstrate the effectiveness of the proposed algorithm.

Application of non-negative matrix factorization to LC/MS data

Liquid Chromatography-Mass Spectrometry (LC/MS) provides large datasets from which one needs to extract the relevant information. Since these data are made of non-negative mixtures of non-negative mass spectra, non-negative matrix factorization (NMF) is well suited for their processing. These data are however very difficult to deal with since they are usually contaminated with non-Gaussian noise and the intensities vary on several orders of magnitude. In this paper, we propose an adaptation of a state-of-the-art NMF algorithms so as to specifically be able to deal with LC/MS data, by using a non-stationary noise model and a stochastic term. We finally perform experiments and compare standard NMF algorithms on both simulated data and an annotated LC/MS dataset. The results of these experiments highlight the significant improvement obtained by our adaptation over other NMF algorithms.

NMF versus ICA for blind source separation

Blind source separation (BSS) is a problem of recovering source signals from signal mixtures without or very limited information about the sources and the mixing process. From literatures, nonnegative matrix factorization (NMF) and independent component analysis (ICA) seem to be the mainstream techniques for solving the BSS problems. Even though the using of NMF and ICA for BSS is well studied, there is still a lack of works that compare the performances of these techniques. Moreover, the nonuniqueness property of NMF is rarely mentioned even though this property actually can make the reconstructed signals vary significantly, and thus introduces the difficulty on how to choose the representative reconstructions from several possible outcomes. In this paper, we compare the performances of NMF and ICA as BSS methods using some standard NMF and ICA algorithms, and point out the difficulty in choosing the representative reconstructions originated from the nonuniqueness property of NMF.

A new discriminant NMF algorithm and its application to the extraction of subtle emotional differences in speech

In this study we propose a new feature extraction algorithm, dNMF (discriminant non-negative matrix factorization), to learn subtle class-related differences while maintaining an accurate generative capability. In addition to the minimum representation error for the standard NMF (non-negative matrix factorization) algorithm, the dNMF algorithm also results in higher between-class variance for discriminant power. The multiplicative NMF learning algorithm has been modified to cope with this additional constraint. The cost function was carefully designed so that the extraction of feature coefficients from a single testing pattern with pre-trained feature vectors resulted in a quadratic convex optimization problem in non-negative space for uniqueness. It also resolves issues related to the previous discriminant NMF algorithms. The developed dNMF algorithm has been applied to the emotion recognition task for speech, where it needs to emphasize the emotional differences while de-emphasizing the dominant phonetic components. The dNMF algorithm successfully extracted subtle emotional differences, demonstrated much better recognition performance and showed a smaller representation error from an emotional speech database.

A flexible R package for nonnegative matrix factorization

BackgroundNonnegative Matrix Factorization (NMF) is an unsupervised learning technique that has been applied successfully in several fields, including signal processing, face recognition and text mining. Recent applications of NMF in bioinformatics have demonstrated its ability to extract meaningful information from high-dimensional data such as gene expression microarrays. Developments in NMF theory and applications have resulted in a variety of algorithms and methods. However, most NMF implementations have been on commercial platforms, while those that are freely available typically require programming skills. This limits their use by the wider research community.ResultsOur objective is to provide the bioinformatics community with an open-source, easy-to-use and unified interface to standard NMF algorithms, as well as with a simple framework to help implement and test new NMF methods. For that purpose, we have developed a package for the R/BioConductor platform. The package ports public code to R, and is structured to enable users to easily modify and/or add algorithms. It includes a number of published NMF algorithms and initialization methods and facilitates the combination of these to produce new NMF strategies. Commonly used benchmark data and visualization methods are provided to help in the comparison and interpretation of the results.ConclusionsThe NMF package helps realize the potential of Nonnegative Matrix Factorization, especially in bioinformatics, providing easy access to methods that have already yielded new insights in many applications. Documentation, source code and sample data are available from CRAN.

Fast Nonnegative Matrix Factorization and Its Application for Protein Fold Recognition

Linear and unsupervised dimensionality reduction via matrix factorization with nonnegativity constraints is studied. Because of these constraints, it stands apart from other linear dimensionality reduction methods. Here we explore nonnegative matrix factorization in combination with three nearest-neighbor classifiers for protein fold recognition. Since typically matrix factorization is iteratively done, convergence, can be slow. To speed up convergence, we perform feature scaling (normalization) prior to the beginning of iterations. This results in a significantly (more than 11 times) faster algorithm. Justification of why it happens is provided. Another modification of the standard nonnegative matrix factorization algorithm is concerned with combining two known techniques for mapping unseen data. This operation is typically necessary before classifying the data in low-dimensional space. Combining two mapping techniques can yield better accuracy than using either technique alone. The gains, however, depend on the state of the random number generator used for initialization of iterations, a classifier, and its parameters. In particular, when employing the best out of three classifiers and reducing the original dimensionality by around 30%, these gains can reach more than 4%, compared to the classification in the original, high-dimensional space.