Non-negative Matrix Factorization Framework Research Articles

Direction of Arrival Based Spatial Covariance Model for Blind Sound Source Separation

This paper addresses the problem of sound source separation from a multichannel microphone array capture via estimation of source spatial covariance matrix (SCM) of a short-time Fourier transformed mixture signal. In many conventional audio separation algorithms the source mixing parameter estimation is done separately for each frequency thus making them prone to errors and leading to suboptimal source estimates. In this paper we propose a SCM model which consists of a weighted sum of direction of arrival (DoA) kernels and estimate only the weights dependent on the source directions. In the proposed algorithm, the spatial properties of the sources become jointly optimized over all frequencies, leading to more coherent source estimates and mitigating the effect of spatial aliasing at high frequencies. The proposed SCM model is combined with a linear model for magnitudes and the parameter estimation is formulated in a complex-valued non-negative matrix factorization (CNMF) framework. Simulations consist of recordings done with a hand-held device sized array having multiple microphones embedded inside the device casing. Separation quality of the proposed algorithm is shown to exceed the performance of existing state of the art separation methods with two sources when evaluated by objective separation quality metrics.

Image clustering by hyper-graph regularized non-negative matrix factorization

Image clustering is a critical step for the applications of content-based image retrieval, image annotation and other high-level image processing. To achieve these tasks, it is essential to obtain proper representation of the images. Non-negative Matrix Factorization (NMF) learns a part-based representation of the data, which is in accordance with how the brain recognizes objects. Due to its psychological and physiological interpretation, NMF has been successfully applied in a wide range of application such as pattern recognition, image processing and computer vision. On the other hand, manifold learning methods discover intrinsic geometrical structure of the high dimension data space. Incorporating manifold regularizer to standard NMF framework leads to novel performance. In this paper, we proposed a novel algorithm, call Hyper-graph regularized Non-negative Matrix Factorization (HNMF) for this purpose. HNMF captures intrinsic geometrical structure by constructing a hyper-graph instead of a simple graph. Hyper-graph model considers high-order relationship of samples and outperforms simple graph model. Empirical experiments demonstrate the effectiveness of the proposed algorithm in comparison to the state-of-the-art algorithms, especially some related works based on NMF.

PROBABILISTIC NON-NEGATIVE MATRIX FACTORIZATION: THEORY AND APPLICATION TO MICROARRAY DATA ANALYSIS

Non-negative matrix factorization (NMF) has proven to be a useful decomposition technique for multivariate data, where the non-negativity constraint is necessary to have a meaningful physical interpretation. NMF reduces the dimensionality of non-negative data by decomposing it into two smaller non-negative factors with physical interpretation for class discovery. The NMF algorithm, however, assumes a deterministic framework. In particular, the effect of the data noise on the stability of the factorization and the convergence of the algorithm are unknown. Collected data, on the other hand, is stochastic in nature due to measurement noise and sometimes inherent variability in the physical process. This paper presents new theoretical and applied developments to the problem of non-negative matrix factorization (NMF). First, we generalize the deterministic NMF algorithm to include a general class of update rules that converges towards an optimal non-negative factorization. Second, we extend the NMF framework to the probabilistic case (PNMF). We show that the Maximum a posteriori (MAP) estimate of the non-negative factors is the solution to a weighted regularized non-negative matrix factorization problem. We subsequently derive update rules that converge towards an optimal solution. Third, we apply the PNMF to cluster and classify DNA microarrays data. The proposed PNMF is shown to outperform the deterministic NMF and the sparse NMF algorithms in clustering stability and classification accuracy.

A New Active Set Method For Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is powerful in data analysis by recovering hidden features from redundant data. Recently, various extensions of NMF have been proposed to further enhance the classification results. In this paper, we present a novel proximal alternating nonnegative least squares (PANLS) framework for NMF. We show the globally convergent property that any accumulation point of PANLS is a stationary point of NMF. A new active set method based on the framework is proposed to solve each subproblem, which reduces to unconstrained optimization after finite steps and converges to the unique minimizer of the subproblem. The PANLS framework and the associated new active set method can be easily extended to variants of NMF. Extensive computational experiments using both synthetic data and real data show that our proposed method is attractive in terms of computational speed, the solution quality, and the flexibility to extensions of NMF.

Determining functional units of tongue motion via graph-regularized sparse non-negative matrix factorization.

Tongue motion during speech and swallowing involves synergies of locally deforming regions, or functional units. Motion clustering during tongue motion can be used to reveal the tongue's intrinsic functional organization. A novel matrix factorization and clustering method for tissues tracked using tagged magnetic resonance imaging (tMRI) is presented. Functional units are estimated using a graph-regularized sparse non-negative matrix factorization framework, learning latent building blocks and the corresponding weighting map from motion features derived from tissue displacements. Spectral clustering using the weighting map is then performed to determine the coherent regions--i.e., functional units--efined by the tongue motion. Two-dimensional image data is used to ver-fy that the proposed algorithm clusters the different types of images ac-urately. Three-dimensional tMRI data from five subjects carrying out simple non-speech/speech tasks are analyzed to show how the proposed approach defines a subject/task-specific functional parcellation of the tongue in localized regions.

Probabilistic spectral envelope modeling of musical instruments within the non-negative matrix factorization framework for mixed music analysis

Non-negative matrix factorization (NMF) has been one of the most useful techniques for musical signal analysis in recent years. In particular, supervised NMF, in which a large number of instrumental samples are used for the analysis, is garnering much attention with respect to analytical accuracy and speed. The accuracy, however, deteriorates if the system does not have enough samples. Therefore, in principle, such methods require as many samples as possible in order for the analysis to be accurate. In this paper, we propose an analysis method that 1) does not require the collection of a large number of training samples, and 2) combines the NMF and probabilistic approaches. In this approach, it is assumed that each instrumental category has a model-invariant feature, called a probabilistic spectral envelope (PSE). As an extension of a spectral envelope, this feature represents the probabilities of spectral envelopes belonging to the instrumental category in a two-dimensional (frequency-amplitude) space. The analysis of an input musical signal is carried out using a supervised NMF framework, where the basis matrix contains the optimum spectra that have been generated from pretrained PSEs.

Non-negative and sparse spectral clustering

Spectral clustering aims to partition a data set into several groups by using the Laplacian of the graph such that data points in the same group are similar while data points in different groups are dissimilar to each other. Spectral clustering is very simple to implement and has many advantages over the traditional clustering algorithms such as k-means. Non-negative matrix factorization (NMF) factorizes a non-negative data matrix into a product of two non-negative (lower rank) matrices so as to achieve dimension reduction and part-based data representation. In this work, we proved that the spectral clustering under some conditions is equivalent to NMF. Unlike the previous work, we formulate the spectral clustering as a factorization of data matrix (or scaled data matrix) rather than the symmetrical factorization of the symmetrical pairwise similarity matrix as the previous study did. Under the NMF framework, where regularization can be easily incorporated into the spectral clustering, we propose several non-negative and sparse spectral clustering algorithms. Empirical studies on real world data show much better clustering accuracy of the proposed algorithms than some state-of-the-art methods such as ratio cut and normalized cut spectral clustering and non-negative Laplacian embedding.

Statistical process monitoring via generalized non-negative matrix projection

As a famous dimension reduction technique, non-negative matrix factorization (NMF) has been used in diverse scientific fields since its appearance. In this work, we aim to propose a new statistical monitoring method based on NMF framework. Considering that the projection method is standardly used in conventional methods such as principal component analysis (PCA), a new variant of NMF method based on positively constrained projections is presented here. This algorithm also relieves the non-negative restriction for original data. Hence it can be called generalized non-negative matrix projection (GNMP). Then, we use GNMP to extract the latent variables that drive a process and to combine them with process monitoring techniques for fault detection. Kernel density estimation (KDE) is adopted to calculate the confidence limits of defined statistical metrics. In addition, corresponding contribution plots are defined for fault isolation. Afterwards, the proposed method is applied to the Tennessee Eastman process to evaluate the monitoring performance. The experiment results clearly illustrate the feasibility of the proposed method.

A nonnegative matrix factorization framework for semi-supervised document clustering with dual constraints

In this paper, we propose a new semi-supervised co-clustering algorithm Orthogonal Semi-Supervised Nonnegative Matrix Factorization (OSS-NMF) for document clustering. In this new approach, the clustering process is carried out by incorporating both prior domain knowledge of data points (documents) in the form of pair-wise constraints and category knowledge of features (words) into the NMF co-clustering framework. Under this framework, the clustering problem is formulated as the problem of finding the local minimizer of objective function, taking into account the dual prior knowledge. The update rules are derived, and an iterative algorithm is designed for the co-clustering process. Theoretically, we prove the correctness and convergence of our algorithm and demonstrate its mathematical rigorous. Our experimental evaluations show that the proposed document clustering model presents remarkable performance improvements with those constraints.

Discovering multirelational structure in social media streams

In this article, we present a novel algorithm to discover multirelational structures from social media streams. A media item such as a photograph exists as part of a meaningful interrelationship among several attributes, including time, visual content, users, and actions. Discovery of such relational structures enables us to understand the semantics of human activity and has applications in content organization, recommendation algorithms, and exploratory social network analysis. We are proposing a novel nonnegative matrix factorization framework to characterize relational structures of group photo streams. The factorization incorporates image content features and contextual information. The idea is to consider a cluster as having similar relational patterns; each cluster consists of photos relating to similar content or context. Relations represent different aspects of the photo stream data, including visual content, associated tags, photo owners, and post times. The extracted structures minimize the mutual information of the predicted joint distribution. We also introduce a relational modularity function to determine the structure cost penalty, and hence determine the number of clusters. Extensive experiments on a large Flickr dataset suggest that our approach is able to extract meaningful relational patterns from group photo streams. We evaluate the utility of the discovered structures through a tag prediction task and through a user study. Our results show that our method based on relational structures, outperforms baseline methods, including feature and tag frequency based techniques, by 35%--420%. We have conducted a qualitative user study to evaluate the benefits of our framework in exploring group photo streams. The study indicates that users found the extracted clustering results clearly represent major themes in a group; the clustering results not only reflect how users describe the group data but often lead the users to discover the evolution of the group activity.

Sparse nonnegative matrix factorization with ℓ0-constraints

Although nonnegative matrix factorization (NMF) favors a sparse and part-based representation of nonnegative data, there is no guarantee for this behavior. Several authors proposed NMF methods which enforce sparseness by constraining or penalizing the ℓ1-norm of the factor matrices. On the other hand, little work has been done using a more natural sparseness measure, the ℓ0-pseudo-norm. In this paper, we propose a framework for approximate NMF which constrains the ℓ0-norm of the basis matrix, or the coefficient matrix, respectively. For this purpose, techniques for unconstrained NMF can be easily incorporated, such as multiplicative update rules, or the alternating nonnegative least-squares scheme. In experiments we demonstrate the benefits of our methods, which compare to, or outperform existing approaches.

A novel computational framework for simultaneous integration of multiple types of genomic data to identify microRNA-gene regulatory modules

Motivation: It is well known that microRNAs (miRNAs) and genes work cooperatively to form the key part of gene regulatory networks. However, the specific functional roles of most miRNAs and their combinatorial effects in cellular processes are still unclear. The availability of multiple types of functional genomic data provides unprecedented opportunities to study the miRNA–gene regulation. A major challenge is how to integrate the diverse genomic data to identify the regulatory modules of miRNAs and genes.Results: Here we propose an effective data integration framework to identify the miRNA–gene regulatory comodules. The miRNA and gene expression profiles are jointly analyzed in a multiple non-negative matrix factorization framework, and additional network data are simultaneously integrated in a regularized manner. Meanwhile, we employ the sparsity penalties to the variables to achieve modular solutions. The mathematical formulation can be effectively solved by an iterative multiplicative updating algorithm. We apply the proposed method to integrate a set of heterogeneous data sources including the expression profiles of miRNAs and genes on 385 human ovarian cancer samples, computationally predicted miRNA–gene interactions, and gene–gene interactions. We demonstrate that the miRNAs and genes in 69% of the regulatory comodules are significantly associated. Moreover, the comodules are significantly enriched in known functional sets such as miRNA clusters, GO biological processes and KEGG pathways, respectively. Furthermore, many miRNAs and genes in the comodules are related with various cancers including ovarian cancer. Finally, we show that comodules can stratify patients (samples) into groups with significant clinical characteristics.Availability: The program and supplementary materials are available at http://zhoulab.usc.edu/SNMNMF/.Contact: xjzhou@usc.edu; zsh@amss.ac.cnSupplementary information: Supplementary data are available at Bioinformatics online.

A non-negative matrix factorization framework for identifying modular patterns in metagenomic profile data

Metagenomic studies sequence DNA directly from environmental samples to explore the structure and function of complex microbial and viral communities. Individual, short pieces of sequenced DNA ("reads") are classified into (putative) taxonomic or metabolic groups which are analyzed for patterns across samples. Analysis of such read matrices is at the core of using metagenomic data to make inferences about ecosystem structure and function. Non-negative matrix factorization (NMF) is a numerical technique for approximating high-dimensional data points as positive linear combinations of positive components. It is thus well suited to interpretation of observed samples as combinations of different components. We develop, test and apply an NMF-based framework to analyze metagenomic read matrices. In particular, we introduce a method for choosing NMF degree in the presence of overlap, and apply spectral-reordering techniques to NMF-based similarity matrices to aid visualization. We show that our method can robustly identify the appropriate degree and disentangle overlapping contributions using synthetic data sets. We then examine and discuss the NMF decomposition of a metabolic profile matrix extracted from 39 publicly available metagenomic samples, and identify canonical sample types, including one associated with coral ecosystems, one associated with highly saline ecosystems and others. We also identify specific associations between pathways and canonical environments, and explore how alternative choices of decompositions facilitate analysis of read matrices at a finer scale.

Non-Negative Matrix Factorization for Semisupervised Heterogeneous Data Coclustering

Coclustering heterogeneous data has attracted extensive attention recently due to its high impact on various important applications, such us text mining, image retrieval, and bioinformatics. However, data coclustering without any prior knowledge or background information is still a challenging problem. In this paper, we propose a Semisupervised Non-negative Matrix Factorization (SS-NMF) framework for data coclustering. Specifically, our method computes new relational matrices by incorporating user provided constraints through simultaneous distance metric learning and modality selection. Using an iterative algorithm, we then perform trifactorizations of the new matrices to infer the clusters of different data types and their correspondence. Theoretically, we prove the convergence and correctness of SS-NMF coclustering and show the relationship between SS-NMF with other well-known coclustering models. Through extensive experiments conducted on publicly available text, gene expression, and image data sets, we demonstrate the superior performance of SS-NMF for heterogeneous data coclustering.

Anatomical Parts-Based Regression Using Non-Negative Matrix Factorization.

Non-negative matrix factorization (NMF) is an excellent tool for unsupervised parts-based learning, but proves to be ineffective when parts of a whole follow a specific pattern. Analyzing such local changes is particularly important when studying anatomical transformations. We propose a supervised method that incorporates a regression constraint into the NMF framework and learns maximally changing parts in the basis images, called Regression based NMF (RNMF). The algorithm is made robust against outliers by learning the distribution of the input manifold space, where the data resides. One of our main goals is to achieve good region localization. By incorporating a gradient smoothing and independence constraint into the factorized bases, contiguous local regions are captured. We apply our technique to a synthetic dataset and structural MRI brain images of subjects with varying ages. RNMF finds the localized regions which are expected to be highly changing over age to be manifested in its significant basis and it also achieves the best performance compared to other statistical regression and dimensionality reduction techniques.

Minimum Dispersion Constrained Nonnegative Matrix Factorization to Unmix Hyperspectral Data

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> This paper considers the problem of unsupervised spectral unmixing for hyperspectral image analysis. Each observed pixel is assumed to be a noisy linear mixture of pure material spectra, namely, <emphasis emphasistype="boldital">endmembers</emphasis>. The mixing coefficients, usually called <emphasis emphasistype="boldital">abundances</emphasis>, are constrained to positive and summed to unity. The proposed unmixing approach is based on the nonnegative matrix factorization (NMF) framework, which considers the physical constraints of the problem, including the positivity of the endmember spectra and abundances. However, the basic NMF formulation has degenerated solutions and suffers from nonconvexity limitations. We consider here a regularization function, called <emphasis emphasistype="boldital">dispersion</emphasis>, which favors the solution such that the endmember spectra have minimum variances. Such a solution encourages the recovered spectra to be flat, preserving the possible spectral singularities (peaks and sharp variations). The regularized criterion is minimized with a projected gradient (PG) scheme, and we propose a new step-size estimation technique to fasten the PG convergence. The derived algorithm is called MiniDisCo, for minimum dispersion constrained NMF. We experimentally compare MiniDisCo with the recently proposed algorithm. It is shown to be particularly robust to the presence of flat spectra, to a possible <emphasis emphasistype="boldital">a priori</emphasis> overestimation of the number of endmembers, or if the amount of observed spectral pixels is low. In addition, experiments show that the considered regularization correctly overcomes the degeneracy and nonconvexity problems, leading to satisfactory unmixing accuracy. We include a comparative analysis of a real-world scene. </para>

Non-negative matrix factorization for semi-supervised data clustering

Traditional clustering algorithms are inapplicable to many real-world problems where limited knowledge from domain experts is available. Incorporating the domain knowledge can guide a clustering algorithm, consequently improving the quality of clustering. In this paper, we propose SS-NMF: a semi-supervised non-negative matrix factorization framework for data clustering. In SS-NMF, users are able to provide supervision for clustering in terms of pairwise constraints on a few data objects specifying whether they “must” or “cannot” be clustered together. Through an iterative algorithm, we perform symmetric tri-factorization of the data similarity matrix to infer the clusters. Theoretically, we show the correctness and convergence of SS-NMF. Moveover, we show that SS-NMF provides a general framework for semi-supervised clustering. Existing approaches can be considered as special cases of it. Through extensive experiments conducted on publicly available datasets, we demonstrate the superior performance of SS-NMF for clustering.

Non-negative matrix factorization framework for dimensionality reduction and unsupervised clustering

Non-negative Matrix Factorization (NMF) is a robust approach to learning spatially localized parts-based subspace patterns in applications such as document analysis, image interpretation, and gene expression analysis. NMF-based decomposition capabilities are lacking in the present ITK toolkit. We provide a generic framework for such decompositions. A specific implementation using a Kulback-Liebler type divergence function is provided to illustrate a possible extension of the base class along with test images to illustrate usage. We have found this method to be robust to noisy image data and show examples from our on-going research using the Allan Brain Atlas data to illustrate its ability to analyze higher dimension data.

Nonnegative features of spectro-temporal sounds for classification

A parts-based representation is a way of understanding object recognition in the brain. The nonnegative matrix factorization (NMF) is an algorithm which is able to learn a parts-based representation by allowing only non-subtractive combinations [Lee, D.D., Seung, H.S., 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791]. In this paper we incorporate a parts-based representation of spectro-temporal sounds into the acoustic feature extraction, which leads to nonnegative features. We present a method of inferring encoding variables in the framework of NMF and show that the method produces robust acoustic features in the presence of noise in the task of general sound classification. Experimental results confirm that the proposed feature extraction method improves the classification performance, especially in the presence of noise, compared to independent component analysis (ICA) which produces holistic features.