Variant Of Nonnegative Matrix Factorization Research Articles

A positive semi definite tensor factorization method for separation of non stationary noise sources

In the aircraft industry, noise mitigation has emerged as an increasingly pressing issue, underscoring the critical importance of advancing our understanding of noise origins within turbofan engines. This paper presents the application of Positive Semi Definite Tensor Factorization (PSDTF), a potential method for the analysis of engine static tests conducted with far-field microphone arrays. By extending the capabilities of Non-negative Matrix Factorization (NMF), PSDTF offers an effective algorithm for source separation. Leveraging on cross spectral matrices to harness phase information across microphones, this approach aims at separating the contributions of several noise sources, avoiding the need for a precise acoustical model (sound propagation, source directivity, etc.). Experimental findings on a controlled experiment demonstrate the superiority of PSDTF over conventional NMF variants in achieving higher-quality source separation.

Alternating Direction Method of Multipliers for Convolutive Non-Negative Matrix Factorization.

Non-negative matrix factorization (NMF) has become a popular method for learning interpretable patterns from data. As one of the variants of standard NMF, convolutive NMF (CNMF) incorporates an extra time dimension to each basis, known as convolutive bases, which is well suited for representing sequential patterns. Previously proposed algorithms for solving CNMF use multiplicative updates which can be derived by either heuristic or majorization-minimization (MM) methods. However, these algorithms suffer from problems, such as low convergence rates, difficulty to reach exact zeroes during iterations and prone to poor local optima. Inspired by the success of alternating direction method of multipliers (ADMMs) on solving NMF, we explore variable splitting (i.e., the core idea of ADMM) for CNMF in this article. New closed-form algorithms of CNMF are derived with the commonly used β -divergences as optimization objectives. Experimental results have demonstrated the efficacy of the proposed algorithms on their faster convergence, better optima, and sparser results than state-of-the-art baselines.

Auto-adjustable hypergraph regularized non-negative matrix factorization for image clustering

Non-negative matrix factorization (NMF) is an effective method for image clustering. However, relatively fixed graph regularization terms and loss functions have been adopted by recently proposed variants of NMF, and their clustering performance can be improved by incorporating configurable parameters. In this paper, an auto-adjustable hypergraph regularized non-negative matrix factorization (AHRNMF) algorithm was proposed. In the AHRNMF framework, we proposed a piecewise loss function and an innovative auto-adjustable hypergraph. The loss function incorporates two adaptive parameters, harmonizing reconstruction error and anti-outlier efficacy. Hypergraph construction relies on the calculation of two k-nearest neighbors (KNN) with different scales. Furthermore, an KNN-based algorithm was developed to assist AHRNMF in achieving auto-adjustment, which can automatically detect outliers without determining the number of clusters in advance. It was demonstrated by extensive experiments that the proposed AHRNMF outperforms other state-of-the-art methods.

Symmetric nonnegative matrix factorization: A systematic review

In recent years, symmetric non-negative matrix factorization (SNMF), a variant of non-negative matrix factorization (NMF), has emerged as a promising tool for data analysis. This paper mainly focuses on the theoretical idea, the basic model, the optimization method, and the variants of SNMF. The SNMF-related approaches can be generally classified into two main categories, Classic SNMFs and Extended SNMFs. The classic SNMFs contain Orthogonal SNMF, Sparse SNMF, Manifold structure based SNMF and Pairwise constraint based SNMF, besides, extended SNMFs include Self-supervised SNMF, MV-WSNMF and Multi-view SNMF. According to different classes of SNMFs, this review elaborates on the key concepts, characteristics, and current issues of these algorithms. The clustering performance of SNMF and its variants on three object image datasets is empirically compared. In addition, the effects of some algorithms for solving SNMF have been compared and the performance of similarity matrix construction methods is also compared. Finally, some open problems with SNMF are discussed.

Unsupervised Pharmaceutical Polymorph Identification and Multicomponent Particle Mapping of ToF-SIMS Data by Non-Negative Matrix Factorization.

Crystal polymorphism of pharmaceutical compounds directly impacts resulting physicochemical characteristics, a critical aspect in active pharmaceutical ingredient (API) production. Tools to characterize and chemically map these polymorphs at the single particle scale remain important to advancing directed manufacture of targeted polymorphs. Here, time-of-flight secondary ion mass spectrometry (ToF-SIMS) was employed for chemically imaging inkjet printed acetaminophen samples. ToF-SIMS generates large data sets of high spatial resolution images. Extracting relevant data and peaks of interest can be laborious for, and biased by, users. Advances in machine learning approaches have introduced many supervised and unsupervised methods for data analysis. In this study, we apply non-negative matrix factorization (NMF) for the unsupervised analysis of ToF-SIMS chemical image data. More specifically, an expanded variant of NMF, NMFk, was employed to determine the data set's latent dimensionality. NMFk combines the spectral unmixing of traditional NMF with k-means clustering of the resulting factors and an optimization of the reconstruction and clustering. The method was used to identify the number of polymorph phases-and their representative mass spectra-generated from inkjet printed acetaminophen samples. Amorphous, crystalline form I, and crystalline form II polymorphs were observed. The learned polymorph mass spectra were then used to map the learned polymorphs onto subsequent particle samples of acetaminophen. Finally, NMFk also enabled the decomposition of mixed particle samples (i.e., migraine medicine), learning the number of compounds and their composition. The extracted constituent phase mass spectra-representing single compounds-were searched against mass spectral libraries for identification.

Analysis of Legal Documents via Non-negative Matrix Factorization Methods

The California Innocence Project (CIP), a clinical law school program aiming to free wrongfully convicted prisoners, evaluates thousands of mails containing new requests for assistance and corresponding case files. Processing and interpreting this large amount of information presents a significant challenge for CIP officials, which can be successfully aided by topic modeling techniques.In this paper, we apply Non-negative Matrix Factorization (NMF) method and implement various offshoots of it to the important and previously unstudied data set compiled by CIP. We identify underlying topics of existing case files and classify request files by crime type and case status (decision type). The results uncover the semantic structure of current case files and can provide CIP officials with a general understanding of newly received case files before further examinations. We also provide an exposition of popular variants of NMF with their experimental results and discuss the benefits and drawbacks of each variant through the real-world application.

Semisupervised Adaptive Symmetric Non-Negative Matrix Factorization.

As a variant of non-negative matrix factorization (NMF), symmetric NMF (SymNMF) can generate the clustering result without additional post-processing, by decomposing a similarity matrix into the product of a clustering indicator matrix and its transpose. However, the similarity matrix in the traditional SymNMF methods is usually predefined, resulting in limited clustering performance. Considering that the quality of the similarity graph is crucial to the final clustering performance, we propose a new semisupervised model, which is able to simultaneously learn the similarity matrix with supervisory information and generate the clustering results, such that the mutual enhancement effect of the two tasks can produce better clustering performance. Our model fully utilizes the supervisory information in the form of pairwise constraints to propagate it for obtaining an informative similarity matrix. The proposed model is finally formulated as a non-negativity-constrained optimization problem. Also, we propose an iterative method to solve it with the convergence theoretically proven. Extensive experiments validate the superiority of the proposed model when compared with nine state-of-the-art NMF models.

Adaptive graph-based discriminative nonnegative matrix factorization for image clustering

Nonnegative matrix factorization(NMF) is an effective dimension reduction method, which is widely used in image clustering and other fields. Some NMF variants preserve the manifold structure of the original data. However, the construction of the traditional neighbor graph depends on the original data, so it may be affected by noise and outliers. Moreover, these methods are unsupervised and do not use available label information. Therefore, this paper presents an adaptive graph-based discriminative nonnegative matrix factorization(AGDNMF). AGDNMF uses the available label to construct the label matrix, such that the new representations with the same label data are aligned to the same axis. And the neighbor graph in AGDNMF is obtained by adaptive iterations. A number of experiments on many image data sets verify that AGDNMF is effective compared with the other state-of-the-art methods.

Off-diagonal symmetric nonnegative matrix factorization

Symmetric nonnegative matrix factorization (symNMF) is a variant of nonnegative matrix factorization (NMF) that allows handling symmetric input matrices and has been shown to be particularly well suited for clustering tasks. In this paper, we present a new model, dubbed off-diagonal symNMF (ODsymNMF), that does not take into account the diagonal entries of the input matrix in the objective function. ODsymNMF has three key advantages compared to symNMF. First, ODsymNMF is theoretically much more sound as there always exists an exact factorization of size at most n(n − 1)/2 where n is the dimension of the input matrix. Second, it makes more sense in practice as diagonal entries of the input matrix typically correspond to the similarity between an item and itself, not bringing much information. Third, it makes the optimization problem much easier to solve. In particular, it will allow us to design an algorithm based on coordinate descent that minimizes the component-wise ℓ1 norm between the input matrix and its approximation. We prove that this norm is much better suited for binary input matrices often encountered in practice. We also derive a coordinate descent method for the component-wise ℓ2 norm, and compare the two approaches with symNMF on synthetic and document datasets.

A Provably Correct and Robust Algorithm for Convolutive Nonnegative Matrix Factorization

In this paper, we propose a provably correct algorithm for convolutive nonnegative matrix factorization (CNMF) under separability assumptions. CNMF is a convolutive variant of nonnegative matrix factorization (NMF), which functions as an NMF with additional sequential structure. This model is useful in a number of applications, such as audio source separation and neural sequence identification. While a number of heuristic algorithms have been proposed to solve CNMF, to the best of our knowledge no provably correct algorithms have been developed. We present an algorithm that takes advantage of the NMF model underlying CNMF and exploits existing algorithms for separable NMF to provably find the unique solution (up to permutation and scaling) under separability-like conditions. Our approach guarantees the solution in low noise settings, and runs in polynomial time. We illustrate its effectiveness on synthetic datasets, and on a singing bird audio sequence.

Near-Convex Archetypal Analysis

Nonnegative matrix factorization (NMF) is a widely used linear dimensionality reduction technique for nonnegative data. NMF requires that each data point is approximated by a convex combination of basis elements. Archetypal analysis (AA), also referred to as convex NMF, is a well-known NMF variant imposing that the basis elements are themselves convex combinations of the data points. AA has the advantage to be more interpretable than NMF because the basis elements are directly constructed from the data points. However, it usually suffers from a high data fitting error because the basis elements are constrained to be contained in the convex cone of the data points. In this letter, we introduce near-convex archetypal analysis (NCAA) which combines the advantages of both AA and NMF. As for AA, the basis vectors are required to be linear combinations of the data points and hence are easily interpretable. As for NMF, the additional flexibility in choosing the basis elements allows NCAA to have a low data fitting error. We show that NCAA compares favorably with a state-of-the-art minimum-volume NMF method on synthetic datasets and on a real-world hyperspectral image.

Discriminative and Orthogonal Subspace Constraints-Based Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is one widely used feature extraction technology in the tasks of image clustering and image classification. For the former task, various unsupervised NMF methods based on the data distribution structure information have been proposed. While for the latter task, the label information of the dataset is one very important guiding. However, most previous proposed supervised NMF methods emphasis on imposing the discriminant constraints on the coefficient matrix. When dealing with new coming samples, the transpose or the pseudoinverse of the basis matrix is used to project these samples to the low dimension space. In this way, the label influence to the basis matrix is indirect. Although, there are also some methods trying to constrain the basis matrix in NMF framework, either they only restrict within-class samples or impose improper constraint on the basis matrix. To address these problems, in this article a novel NMF framework named discriminative and orthogonal subspace constraints-based nonnegative matrix factorization (DOSNMF) is proposed. In DOSNMF, the discriminative constraints are imposed on the projected subspace instead of the directly learned representation. In this manner, the discriminative information is directly connected with the projected subspace. At the same time, an orthogonal term is incorporated in DOSNMF to adjust the orthogonality of the learned basis matrix, which can ensure the orthogonality of the learned subspace and improve the sparseness of the basis matrix at the same time. This framework can be implemented in two ways. The first way is based on the manifold learning theory. In this way, two graphs, i.e., the intrinsic graph and the penalty graph, are constructed to capture the intra-class structure and the inter-class distinctness. With this design, both the manifold structure information and the discriminative information of the dataset are utilized. For convenience, we name this method as the name of the framework, i.e., DOSNMF. The second way is based on the Fisher’s criterion, we name it Fisher’s criterion-based DOSNMF (FDOSNMF). The objective functions of DOSNMF and FDOSNMF can be easily optimized using multiplicative update (MU) rules. The new methods are tested on five datasets and compared with several supervised and unsupervised variants of NMF. The experimental results reveal the effectiveness of the proposed methods.

Road traffic sound level estimation from realistic urban sound mixtures by Non-negative Matrix Factorization

Experimental acoustic sensor networks are currently tested in large cities, and appear more and more as a useful tool to enrich modeled road traffic noise maps through data assimilation techniques. One challenge is to be able to isolate from the measured sound mixtures acoustic quantities of interest such as the sound level of road traffic. This task is anything but trivial because of the multiple sound sources that overlap within urban sound mixtures.In this paper, the Non-negative Matrix Factorization (NMF) framework is developed to estimate road traffic noise levels within urban sound scenes. To evaluate the performances of the proposed approach, a synthetic corpus of sound scenes is designed, to cover most common soundscape settings, and whom realism is validated through a perceptual test. The simulated scenes reproduce then the sensor network outputs, in which the actual occurrence and sound level of each source are known.Several variants of NMF are tested. The proposed approach, named threshold initialized NMF, appears to be the most reliable approach, allowing road traffic noise level estimation with average errors of less than 1.3 dB over the tested corpus of sound scenes.

Topological structure regularized nonnegative matrix factorization for image clustering

Image representation is supposed to reveal the distinguishable feature and be captured in unsupervised fashion. Recently, nonnegative matrix factorization (NMF) has been widely used in capturing the parts-based feature for image representation. Previous NMF variants for image clustering first set the reduced rank as the number of clusters to obtain image representations. Then, K-means technology is adopted for ultimate clustering. However, the setting of rank is too rigid to preserve the geometrical structure of images, leading to unsuitable representation for the following clustering. Moreover, the image representation learning and clustering procedures are conducted individually which is inefficient as well as suboptimal for image clustering. This paper incorporates the constraint of topological structure into the procedure of NMF for simultaneously accomplishing nonnegative image representation and image clustering task. The proposed topological structure regularized NMF (TNMF) makes the rank bigger than the number of clusters, allotting one image more than one coarse clusters to guarantee the geometrical structure of images. In addition, a connected graph constructed by the coefficient matrix is enforced to have the strong connected components whose number equals to the number of clusters. By checking the number of connected components of graph against the number of clusters during the learning procedure, the optimization of TNMF is accomplished using alternative update rules. Extensive experiments on the challenging face, digit, and object data sets demonstrate the effectiveness of TNMF compared with competitive NMF variants for image clustering.

Pairwise Constraint Propagation-Induced Symmetric Nonnegative Matrix Factorization.

As a variant of nonnegative matrix factorization (NMF), symmetric NMF (SNMF) has shown to be effective for capturing the cluster structure embedded in the graph representation. In contrast to the existing SNMF-based clustering methods that empirically construct the similarity matrix and rigidly introduce the supervisory information to the assignment matrix, in this paper, we propose a novel SNMF-based semisupervised clustering method, namely, pairwise constraint propagation-induced SNMF (PCPSNMF). By formulating a single-constrained optimization problem, PCPSNMF is capable of learning the similarity and assignment matrices adaptively and simultaneously, in which a small amount of supervisory information in the form of pairwise constraints is introduced in a flexible way to guide the construction of the similarity matrix, and the two matrices communicate with each other to achieve mutual refinement until convergence. In addition, we propose an efficient alternating iterative algorithm to solve the optimization problem, whose convergence is theoretically proven. Experimental results over several benchmark image data sets demonstrate that PCPSNMF is less sensitive to initialization and produces higher clustering performance, compared with the state-of-the-art methods.

CoDiNMF: Co-Clustering of Directed Graphs via NMF

Co-clustering computes clusters of data items and the related features concurrently, and it has been used in many applications such as community detection, product recommendation, computer vision, and pricing optimization. In this paper, we propose a new co-clustering method, called CoDiNMF, which improves the clustering quality and finds directional patterns among co-clusters by using multiple directed and undirected graphs. We design the objective function of co-clustering by using min-cut criterion combined with an additional term which controls the sum of net directional flow between different co-clusters. In addition, we show that a variant of Nonnegative Matrix Factorization (NMF) can solve the proposed objective function effectively. We run experiments on the US patents and BlogCatalog data sets whose ground truth have been known, and show that CoDiNMF improves clustering results compared to other co-clustering methods in terms of average F1 score, Rand index, and adjusted Rand index (ARI). Finally, we compare CoDiNMF and other co-clustering methods on the Wikipedia data set of philosophers, and we can find meaningful directional flow of influence among co-clusters of philosophers.

Simultaneous Spectral Data Embedding and Clustering.

Spectral clustering is often carried out by combining spectral data embedding and -means clustering. However, the aims, dimensionality reduction and clustering, are usually not performed jointly. In this brief, we propose a novel approach to finding an optimal spectral embedding for identifying a partition of the set of objects; it iteratively alternates spectral embedding and clustering. In doing so, we show that our model can learn a low-dimensional representation more suited to clustering. Compared with classical spectral clustering methods, the proposed algorithm is not costly and outperforms not only these methods but also other nonnegative matrix factorization variants.

Label and orthogonality regularized non-negative matrix factorization for image classification

As one of the most popular data-representation methods, non-negative matrix factorization (NMF) has been widely used in image processing and pattern recognition. Compared with other dimension reduction methods, we can interpret the data with psychological intuition using NMF since NMF can decompose the whole into visual parts by learning the non-negative basis. However, the original NMF lacks of extracting the discriminant information of the data for the image classification task. For enhancing the discriminant and parts-based interpretability, this work proposes a label and orthogonality regularized NMF (LONMF) algorithm based on the squared Euclidean distance. LONMF takes into account the label consistence with the low-dimensional projected data and orthogonal property of the non-negative basis. By integrating the non-negative constraint, label consistence, and orthogonal property into the objective function, the efficient updating procedure can obtain a discriminant basis matrix. Meanwhile, we design a linear classifier using the projected data to guide the label for efficient image classification task. Experiment results of the competitive NMF variants on the challenging digit and face databases demonstrate the effectiveness of the proposed LONMF algorithm.

Joint Linear Regression and Nonnegative Matrix Factorization Based on Self-Organized Graph for Image Clustering and Classification

Nonnegative matrix factorization (NMF) technique has been developed successfully to represent the intuitively meaningful feature of data. A suitable representation can faithfully preserve the intrinsic structure of data. Due to the fact that it introduces the label information, semi-supervised NMF has been demonstrated more advantageous in image representation than original NMF. However, previous semi-supervised NMF variants construct a label indicator matrix only for tagging the labeled data and not being optimized together with the matrix factorization. It is short of label propagation and fails to work for predicting the attribution of data. Moreover, the transductive semi-supervised NMF variants cannot dispose the prediction of unseen data, restricting the application of NMF. In this paper, a joint optimization framework of linear regression and NMF (LR-NMF) based on the self-organized graph is proposed for a completed task which simultaneously takes into account image representation and attribution prediction. By minimizing the proposed objective, three interactive threads are running: decomposing the data into nonnegative basis matrix and the corresponding representation, linear regression using the nonnegative representation, and label propagation based on the self-organized graph which is defined in the feature space. The products of LR-NMF can be viewed as extracting nonnegative feature for clustering, meanwhile, they can be used to solve the out-of-sample problem for classification. Extensive clustering and classification experiments on the digit, face, and object challenging data sets are presented to show the efficacy of the proposed LR-NMF algorithm.

Multiplicative updates for polynomial root finding

Let f(x)=p(x)−q(x) be a polynomial with real coefficients whose roots have nonnegative real part, where p and q are polynomials with nonnegative coefficients. In this paper, we prove the following: Given an initial point x0>0, the multiplicative update xt+1=xtp(xt)/q(xt) (t=0,1,…) monotonically and linearly converges to the largest (resp. smallest) real roots of f smaller (resp. larger) than x0 if p(x0)<q(x0) (resp. q(x0)<p(x0)). The motivation to study this algorithm comes from the multiplicative updates proposed in the literature to solve optimization problems with nonnegativity constraints; in particular many variants of nonnegative matrix factorization.