Regularized Non-negative Matrix Factorization Research Articles

Manifold ranking graph regularization non-negative matrix factorization with global and local structures

Non-negative matrix factorization (NMF) is a recently popularized technique for learning parts-based, linear representations of non-negative data. Although the decomposition rate of NMF is very fast, it still suffers from the following deficiency: It only revealed the local geometry structure; global geometric information of data set is ignored. This paper proposes a manifold ranking graph regularization non-negative matrix factorization with local and global geometric structure (MRLGNMF) to overcome the above deficiency. In particular, MRLGNMF induces manifold ranking to the non-negative matrix factorization with Sinkhorn distance. Numerical results show that the new algorithm is superior to the existing algorithm.

A Unified Convergence Analysis of the Multiplicative Update Algorithm for Regularized Nonnegative Matrix Factorization

The multiplicative update (MU) algorithm has been extensively used to estimate the basis and coefficient matrices in nonnegative matrix factorization (NMF) problems under a wide range of divergences and regularizers. However, theoretical convergence guarantees have only been derived for a few special divergences without regularization. In this work, we provide a conceptually simple, self-contained, and unified proof for the convergence of the MU algorithm applied on NMF with a wide range of divergences and regularizers. Our main result shows the sequence of iterates (i.e., pairs of basis and coefficient matrices) produced by the MU algorithm converges to the set of stationary points of the nonconvex NMF optimization problem. Our proof strategy has the potential to open up new avenues for analyzing similar problems in machine learning and signal processing.

Refined-Graph Regularization-Based Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is one of the most popular data representation methods in the field of computer vision and pattern recognition. High-dimension data are usually assumed to be sampled from the submanifold embedded in the original high-dimension space. To preserve the locality geometric structure of the data, k -nearest neighbor ( k -NN) graph is often constructed to encode the near-neighbor layout structure. However, k -NN graph is based on Euclidean distance, which is sensitive to noise and outliers. In this article, we propose a refined-graph regularized nonnegative matrix factorization by employing a manifold regularized least-squares regression (MRLSR) method to compute the refined graph. In particular, each sample is represented by the whole dataset regularized with ℓ 2 -norm and Laplacian regularizer. Then a MRLSR graph is constructed based on the representative coefficients of each sample. Moreover, we present two optimization schemes to generate refined-graphs by employing a hard-thresholding technique. We further propose two refined-graph regularized nonnegative matrix factorization methods and use them to perform image clustering. Experimental results on several image datasets reveal that they outperform 11 representative methods.

Semi-automated brain tumor segmentation on multi-parametric MRI using regularized non-negative matrix factorization

BackgroundSegmentation of gliomas in multi-parametric (MP-)MR images is challenging due to their heterogeneous nature in terms of size, appearance and location. Manual tumor segmentation is a time-consuming task and clinical practice would benefit from (semi-) automated segmentation of the different tumor compartments.MethodsWe present a semi-automated framework for brain tumor segmentation based on non-negative matrix factorization (NMF) that does not require prior training of the method. L1-regularization is incorporated into the NMF objective function to promote spatial consistency and sparseness of the tissue abundance maps. The pathological sources are initialized through user-defined voxel selection. Knowledge about the spatial location of the selected voxels is combined with tissue adjacency constraints in a post-processing step to enhance segmentation quality. The method is applied to an MP-MRI dataset of 21 high-grade glioma patients, including conventional, perfusion-weighted and diffusion-weighted MRI. To assess the effect of using MP-MRI data and the L1-regularization term, analyses are also run using only conventional MRI and without L1-regularization. Robustness against user input variability is verified by considering the statistical distribution of the segmentation results when repeatedly analyzing each patient’s dataset with a different set of random seeding points.ResultsUsing L1-regularized semi-automated NMF segmentation, mean Dice-scores of 65%, 74 and 80% are found for active tumor, the tumor core and the whole tumor region. Mean Hausdorff distances of 6.1 mm, 7.4 mm and 8.2 mm are found for active tumor, the tumor core and the whole tumor region. Lower Dice-scores and higher Hausdorff distances are found without L1-regularization and when only considering conventional MRI data.ConclusionsBased on the mean Dice-scores and Hausdorff distances, segmentation results are competitive with state-of-the-art in literature. Robust results were found for most patients, although careful voxel selection is mandatory to avoid sub-optimal segmentation.

Image annotation using multi-view non-negative matrix factorization with different number of basis vectors

Automatic Image Annotation (AIA) helps image retrieval systems by predicting tags for images. In this paper, we propose an AIA system using Non-negative Matrix Factorization (NMF) framework. The NMF framework discovers a latent space, by factorizing data into a set of non-negative basis and coefficients. To model the images, multiple features are extracted, each one represents images from a specific view. We use multi-view graph regularization NMF and allow NMF to choose a different number of basis vectors for each view. For tag prediction, each test image is mapped onto the multiple latent spaces. The distances of images in these spaces are used to form a unified distance matrix. The weights of distances are learned automatically. Then a search-based method is used to predict tags based on tags of nearest neighbors’. We evaluate our method on three datasets and show that it is competitive with the current state-of-the-art methods.

Multi-view clustering via multi-manifold regularized non-negative matrix factorization

Non-negative matrix factorization based multi-view clustering algorithms have shown their competitiveness among different multi-view clustering algorithms. However, non-negative matrix factorization fails to preserve the locally geometrical structure of the data space. In this paper, we propose a multi-manifold regularized non-negative matrix factorization framework (MMNMF) which can preserve the locally geometrical structure of the manifolds for multi-view clustering. MMNMF incorporates consensus manifold and consensus coefficient matrix with multi-manifold regularization to preserve the locally geometrical structure of the multi-view data space. We use two methods to construct the consensus manifold and two methods to find the consensus coefficient matrix, which leads to four instances of the framework. Experimental results show that the proposed algorithms outperform existing non-negative matrix factorization based algorithms for multi-view clustering.

ℓ1/2-norm regularized nonnegative low-rank and sparse affinity graph for remote sensing image segmentation

Segmentation of real-world remote sensing images is a challenge due to the complex texture information with high heterogeneity. Thus, graph-based image segmentation methods have been attracting great attention in the field of remote sensing. However, most of the traditional graph-based approaches fail to capture the intrinsic structure of the feature space and are sensitive to noises. A l1/2-norm regularization-based graph segmentation method is proposed to segment remote sensing images. First, we use the occlusion of the random texture model (ORTM) to extract the local histogram features. Then, a l1/2-norm regularized low-rank and sparse representation (L1/2NNLRS) is implemented to construct a l1/2-regularized nonnegative low-rank and sparse graph (L1/2NNLRS-graph), by the union of feature subspaces. Moreover, the L1/2NNLRS-graph has a high ability to discriminate the manifold intrinsic structure of highly homogeneous texture information. Meanwhile, the L1/2NNLRS representation takes advantage of the low-rank and sparse characteristics to remove the noises and corrupted data. Last, we introduce the L1/2NNLRS-graph into the graph regularization nonnegative matrix factorization to enhance the segmentation accuracy. The experimental results using remote sensing images show that when compared to five state-of-the-art image segmentation methods, the proposed method achieves more accurate segmentation results.

Efficient Nonnegative Matrix Factorization by DC Programming and DCA.

In this letter, we consider the nonnegative matrix factorization (NMF) problem and several NMF variants. Two approaches based on DC (difference of convex functions) programming and DCA (DC algorithm) are developed. The first approach follows the alternating framework that requires solving, at each iteration, two nonnegativity-constrained least squares subproblems for which DCA-based schemes are investigated. The convergence property of the proposed algorithm is carefully studied. We show that with suitable DC decompositions, our algorithm generates most of the standard methods for the NMF problem. The second approach directly applies DCA on the whole NMF problem. Two algorithms-one computing all variables and one deploying a variable selection strategy-are proposed. The proposed methods are then adapted to solve various NMF variants, including the nonnegative factorization, the smooth regularization NMF, the sparse regularization NMF, the multilayer NMF, the convex/convex-hull NMF, and the symmetric NMF. We also show that our algorithms include several existing methods for these NMF variants as special versions. The efficiency of the proposed approaches is empirically demonstrated on both real-world and synthetic data sets. It turns out that our algorithms compete favorably with five state-of-the-art alternating nonnegative least squares algorithms.

Protein functional properties prediction in sparsely-label PPI networks through regularized non-negative matrix factorization.

BackgroundPredicting functional properties of proteins in protein-protein interaction (PPI) networks presents a challenging problem and has important implication in computational biology. Collective classification (CC) that utilizes both attribute features and relational information to jointly classify related proteins in PPI networks has been shown to be a powerful computational method for this problem setting. Enabling CC usually increases accuracy when given a fully-labeled PPI network with a large amount of labeled data. However, such labels can be difficult to obtain in many real-world PPI networks in which there are usually only a limited number of labeled proteins and there are a large amount of unlabeled proteins. In this case, most of the unlabeled proteins may not connected to the labeled ones, the supervision knowledge cannot be obtained effectively from local network connections. As a consequence, learning a CC model in sparsely-labeled PPI networks can lead to poor performance.ResultsWe investigate a latent graph approach for finding an integration latent graph by exploiting various latent linkages and judiciously integrate the investigated linkages to link (separate) the proteins with similar (different) functions. We develop a regularized non-negative matrix factorization (RNMF) algorithm for CC to make protein functional properties prediction by utilizing various data sources that are available in this problem setting, including attribute features, latent graph, and unlabeled data information. In RNMF, a label matrix factorization term and a network regularization term are incorporated into the non-negative matrix factorization (NMF) objective function to seek a matrix factorization that respects the network structure and label information for classification prediction.ConclusionExperimental results on KDD Cup tasks predicting the localization and functions of proteins to yeast genes demonstrate the effectiveness of the proposed RNMF method for predicting the protein properties. In the comparison, we find that the performance of the new method is better than those of the other compared CC algorithms especially in paucity of labeled proteins.

Automatic segmentation of odor maps in the mouse olfactory bulb using regularized non-negative matrix factorization

Segmentation of functional parts in image series of functional activity is a common problem in neuroscience. Here we apply regularized non-negative matrix factorization (rNMF) to extract glomeruli in intrinsic optical signal (IOS) images of the olfactory bulb. Regularization allows us to incorporate prior knowledge about the spatio-temporal characteristics of glomerular signals. We demonstrate how to identify suitable regularization parameters on a surrogate dataset. With appropriate regularization segmentation by rNMF is more resilient to noise and requires fewer observations than conventional spatial independent component analysis (sICA). We validate our approach in experimental data using anatomical outlines of glomeruli obtained by 2-photon imaging of resting synapto-pHluorin fluorescence. Taken together, we show that rNMF provides a straightforward method for problem tailored source separation that enables reliable automatic segmentation of functional neural images, with particular benefit in situations with low signal-to-noise ratio as in IOS imaging.

Co-clustering over multiple dynamic data streams based on non-negative matrix factorization

Clustering multiple data streams has become an active area of research with many practical applications. Most of the early work in this area focused on one-sided clustering, i.e., clustering data streams based on feature correlation. However, recent research has shown that data streams can be grouped based on the distribution of their features, while features can be grouped based on their distribution across data streams. In this paper, an evolutionary clustering algorithm is proposed for multiple data streams using graph regularization non-negative matrix factorization (EC-NMF) in which the geometric structure of both the data and feature manifold is considered. Instead of directly clustering multiple data streams periodically, EC-NMF works in the low-rank approximation subspace and incorporates prior knowledge from historic results with temporal smoothness. Furthermore, we develop an iterative algorithm and provide convergence and correctness proofs from a theoretical standpoint. The effectiveness and efficiency of the algorithm are both demonstrated in experiments on real and synthetic data sets. The results show that the proposed EC-NMF algorithm outperforms existing methods for clustering multiple data streams evolving over time.

Source separation using regularized NMF with MMSE estimates under GMM priors with online learning for the uncertainties

We propose a new method to incorporate priors on the solution of nonnegative matrix factorization (NMF). The NMF solution is guided to follow the minimum mean square error (MMSE) estimates of the weight combinations under a Gaussian mixture model (GMM) prior. The proposed algorithm can be used for denoising or single-channel source separation (SCSS) applications. NMF is used in SCSS in two main stages, the training stage and the separation stage. In the training stage, NMF is used to decompose the training data spectrogram for each source into a multiplication of a trained basis and gains matrices. In the separation stage, the mixed signal spectrogram is decomposed as a weighted linear combination of the trained basis matrices for the source signals. In this work, to improve the separation performance of NMF, the trained gains matrices are used to guide the solution of the NMF weights during the separation stage. The trained gains matrix is used to train a prior GMM that captures the statistics of the valid weight combinations that the columns of the basis matrix can receive for a given source signal. In the separation stage, the prior GMMs are used to guide the NMF solution of the gains/weights matrices using MMSE estimation. The NMF decomposition weights matrix is treated as a distorted image by a distortion operator, which is learned directly from the observed signals. The MMSE estimate of the weights matrix under the trained GMM prior and log-normal distribution for the distortion is then found to improve the NMF decomposition results. The MMSE estimate is embedded within the optimization objective to form a novel regularized NMF cost function. The corresponding update rules for the new objectives are derived in this paper. The proposed MMSE estimates based regularization avoids the problem of computing the hyper-parameters and the regularization parameters. MMSE also provides a better estimate for the valid gains matrix. Experimental results show that the proposed regularized NMF algorithm improves the source separation performance compared with using NMF without a prior or with other prior models.

Image clustering by hyper-graph regularized non-negative matrix factorization

Image clustering is a critical step for the applications of content-based image retrieval, image annotation and other high-level image processing. To achieve these tasks, it is essential to obtain proper representation of the images. Non-negative Matrix Factorization (NMF) learns a part-based representation of the data, which is in accordance with how the brain recognizes objects. Due to its psychological and physiological interpretation, NMF has been successfully applied in a wide range of application such as pattern recognition, image processing and computer vision. On the other hand, manifold learning methods discover intrinsic geometrical structure of the high dimension data space. Incorporating manifold regularizer to standard NMF framework leads to novel performance. In this paper, we proposed a novel algorithm, call Hyper-graph regularized Non-negative Matrix Factorization (HNMF) for this purpose. HNMF captures intrinsic geometrical structure by constructing a hyper-graph instead of a simple graph. Hyper-graph model considers high-order relationship of samples and outperforms simple graph model. Empirical experiments demonstrate the effectiveness of the proposed algorithm in comparison to the state-of-the-art algorithms, especially some related works based on NMF.

PROBABILISTIC NON-NEGATIVE MATRIX FACTORIZATION: THEORY AND APPLICATION TO MICROARRAY DATA ANALYSIS

Non-negative matrix factorization (NMF) has proven to be a useful decomposition technique for multivariate data, where the non-negativity constraint is necessary to have a meaningful physical interpretation. NMF reduces the dimensionality of non-negative data by decomposing it into two smaller non-negative factors with physical interpretation for class discovery. The NMF algorithm, however, assumes a deterministic framework. In particular, the effect of the data noise on the stability of the factorization and the convergence of the algorithm are unknown. Collected data, on the other hand, is stochastic in nature due to measurement noise and sometimes inherent variability in the physical process. This paper presents new theoretical and applied developments to the problem of non-negative matrix factorization (NMF). First, we generalize the deterministic NMF algorithm to include a general class of update rules that converges towards an optimal non-negative factorization. Second, we extend the NMF framework to the probabilistic case (PNMF). We show that the Maximum a posteriori (MAP) estimate of the non-negative factors is the solution to a weighted regularized non-negative matrix factorization problem. We subsequently derive update rules that converge towards an optimal solution. Third, we apply the PNMF to cluster and classify DNA microarrays data. The proposed PNMF is shown to outperform the deterministic NMF and the sparse NMF algorithms in clustering stability and classification accuracy.

Regularized nonnegative matrix factorization using Gaussian mixture priors for supervised single channel source separation

We introduce a new regularized nonnegative matrix factorization (NMF) method for supervised single-channel source separation (SCSS). We propose a new multi-objective cost function which includes the conventional divergence term for the NMF together with a prior likelihood term. The first term measures the divergence between the observed data and the multiplication of basis and gains matrices. The novel second term encourages the log-normalized gain vectors of the NMF solution to increase their likelihood under a prior Gaussian mixture model (GMM) which is used to encourage the gains to follow certain patterns. In this model, the parameters to be estimated are the basis vectors, the gain vectors and the parameters of the GMM prior. We introduce two different ways to train the model parameters, sequential training and joint training. In sequential training, after finding the basis and gains matrices, the gains matrix is then used to train the prior GMM in a separate step. In joint training, within each NMF iteration the basis matrix, the gains matrix and the prior GMM parameters are updated jointly using the proposed regularized NMF. The normalization of the gains makes the prior models energy independent, which is an advantage as compared to earlier proposals. In addition, GMM is a much richer prior than the previously considered alternatives such as conjugate priors which may not represent the distribution of the gains in the best possible way. In the separation stage after observing the mixed signal, we use the proposed regularized cost function with a combined basis and the GMM priors for all sources that were learned from training data for each source. Only the gain vectors are estimated from the mixed data by minimizing the joint cost function. We introduce novel update rules that solve the optimization problem efficiently for the new regularized NMF problem. This optimization is challenging due to using energy normalization and GMM for prior modeling, which makes the problem highly nonlinear and non-convex. The experimental results show that the introduced methods improve the performance of single channel source separation for speech separation and speech–music separation with different NMF divergence functions. The experimental results also show that, using the GMM prior gives better separation results than using the conjugate prior.

Graph dual regularization non-negative matrix factorization for co-clustering

Low-rank matrix factorization is one of the most useful tools in scientific computing, data mining and computer vision. Among of its techniques, non-negative matrix factorization (NMF) has received considerable attention due to producing a parts-based representation of the data. Recent research has shown that not only the observed data are found to lie on a nonlinear low dimensional manifold, namely data manifold, but also the features lie on a manifold, namely feature manifold. In this paper, we propose a novel algorithm, called graph dual regularization non-negative matrix factorization (DNMF), which simultaneously considers the geometric structures of both the data manifold and the feature manifold. We also present a graph dual regularization non-negative matrix tri-factorization algorithm (DNMTF) as an extension of DNMF. Moreover, we develop two iterative updating optimization schemes for DNMF and DNMTF, respectively, and provide the convergence proofs of our two optimization schemes. Experimental results on UCI benchmark data sets, several image data sets and a radar HRRP data set demonstrate the effectiveness of both DNMF and DNMTF. ► Graph dual regularization non-negative matrix factorization algorithm is proposed. ► Graph dual regularization non-negative matrix tri-factorization algorithm is proposed. ► DNMF and DNMTF simultaneously consider geometric structures of data and features. ► Two iterative multiplicative updating optimization schemes are presented. ► The convergence proofs of two optimization schemes are provided.

Extraction of functional domains in optical imaging data using regularized nonnegative matrix factorization

Event Abstract Back to Event Extraction of functional domains in optical imaging data using regularized nonnegative matrix factorization Jan Soelter1*, Jan Schumacher2, Antonia Strutz3, Silke Sachse3, Hartwig Spors2 and Michael Schmuker1, 4 1 Freie Universität Berlin, Institute of Biology, Germany 2 Max Planck Institute of Biophysics, Department of Molecular Neurogenetics, Germany 3 Max Planck Institute for Chemical Ecology, Department of Evolutionary Neuroethology, Germany 4 Bernstein Center for Computational Neuroscience Berlin, Germany Functional optical imaging is widely used to measure neuronal activity in the living brain. In many cases, the neuronal activity exhibits spatial structure which reflects a distinct physiological organization of the tissue. However, exact data on that organization is often not available in functional imaging. This poses the challenge of automatically estimating distinct spatial domains and their corresponding stimulus response based on functional data alone. The decomposition of observations into their generating sources is an instance of the Blind Source Separation problem. Previous approaches to this problem were based on Principal and Independent Component Analysis. However, both methods reach their limits if the number of sources lies within the range of the number of images, or even exceeds them. In this contribution, we introduce a regularized non-negative matrix factorization algorithm to overcome this problem. Using a toy dataset, we compare its performance to established approaches. In addition, we demonstrate its applicability in two natural datasets where the non-negativity condition is met: to discover (a) glomeruli, the functional domains of the olfactory bulb in mice, in intrinsic optical images and (b) inhibitory olfactory projection neurons in calcium imaging data of the lateral horn of Drosophila. To cope with the low observation to source ratio, we restricted the generative model in the introduced algorithm in several ways. The non-negative matrix factorization itself imposed the constraint of only positive contributions of the sources. Moreover, a sparseness constraint enforced localized sources. Additionally, we introduced a new modification in the Hierarchical Alternating Least Squares (HALS) algorithm to promote spatially decorrelated sources. Acknowledgements Funded by DFG grant no. SCHM2474/1-1 to M.S. (SPP 1392, principal source of funding), BMBF grant no. 01GQ1001D (BCCN Berlin) to M.S., BMBF grant no. 01GQ0807 to S.S., DFG grant no. SP 1134/2-1to H.S. (SPP 1392). Keywords: calcium imaging, data analysis and machine learning, Drosophila lateral horn, Intrinsic signal imaging, mouse olfactory bulb Conference: BC11 : Computational Neuroscience & Neurotechnology Bernstein Conference & Neurex Annual Meeting 2011, Freiburg, Germany, 4 Oct - 6 Oct, 2011. Presentation Type: Poster Topic: data analysis and machine learning (please use "data analysis and machine learning" as keyword) Citation: Soelter J, Schumacher J, Strutz A, Sachse S, Spors H and Schmuker M (2011). Extraction of functional domains in optical imaging data using regularized nonnegative matrix factorization. Front. Comput. Neurosci. Conference Abstract: BC11 : Computational Neuroscience & Neurotechnology Bernstein Conference & Neurex Annual Meeting 2011. doi: 10.3389/conf.fncom.2011.53.00077 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 23 Aug 2011; Published Online: 04 Oct 2011. * Correspondence: Mr. Jan Soelter, Freie Universität Berlin, Institute of Biology, Berlin, 14195, Germany, j.soelter@fu-berlin.de Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Jan Soelter Jan Schumacher Antonia Strutz Silke Sachse Hartwig Spors Michael Schmuker Google Jan Soelter Jan Schumacher Antonia Strutz Silke Sachse Hartwig Spors Michael Schmuker Google Scholar Jan Soelter Jan Schumacher Antonia Strutz Silke Sachse Hartwig Spors Michael Schmuker PubMed Jan Soelter Jan Schumacher Antonia Strutz Silke Sachse Hartwig Spors Michael Schmuker Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Nonnegative matrix factorization: a blind spectra separation method for <italic>in vivo</italic> fluorescent optical imaging

Fluorescence imaging in diffusive media is an emerging imaging modality for medical applications that uses injected fluorescent markers that bind to specific targets, e.g., carcinoma. The region of interest is illuminated with near-IR light and the emitted back fluorescence is analyzed to localize the fluorescence sources. To investigate a thick medium, as the fluorescence signal decreases with the light travel distance, any disturbing signal, such as biological tissues intrinsic fluorescence (called autofluorescence) is a limiting factor. Several specific markers may also be simultaneously injected to bind to different molecules, and one may want to isolate each specific fluorescent signal from the others. To remove the unwanted fluorescence contributions or separate different specific markers, a spectroscopic approach is explored. The nonnegative matrix factorization (NMF) is the blind positive source separation method we chose. We run an original regularized NMF algorithm we developed on experimental data, and successfully obtain separated in vivo fluorescence spectra.