Original Non-negative Matrix Factorization Research Articles

Structural regulation of Mn-based Prussian blue induced by zinc-substitution for enhanced sodium storage performance

Mn-based Prussian blue (NMF) is a prospective choice for sodium-ion battery cathode materials because of its high theoretical specific capacity, high charge/discharge voltage plateau, cheap raw materials and eco-friendly. However, it undergoes serious capacity fading during cycling owing to the unstable Jahn-Teller distortion caused by the oxidation of Mn2+ to Mn3+. In this work, the structural regulation of NMF is achieved through zinc substitution, and the effect of structural adjustment caused by zinc substitution on the sodium storage properties of NMF is studied in detail. The experimental results verify that NZMF still maintain cubic structure with high content of sodium, less [Fe(CN)6] vacancies and low level of crystal water. Moreover, the four coordination of Zn-N is lower than six coordination of Mn-N, confining the nucleus growth, thereby reducing the powders size of NZMF. The smaller powder size of NZMF with higher surface area results in shortened sodium ion diffusion path and accelerated the charge transfer. In addition, the electrochemically inactive zinc plays a pillar role in maintaining its structural stability. Consequently, NZMF reveals better rate capacity and cycling stability than original NMF. In particular, its capacity can be maintained at 68.6 mAh g−1 under a high current density of 1600 mA/g, and the capacity retention is 76.5% at 800 mA/g after 1000 cycles. This work provides a new insight to improve the structural stability of Prussian blue.

SDNMF: Semisupervised discriminative nonnegative matrix factorization for feature learning

As one of the most effective feature learning methods, Nonnegative Matrix Factorization (NMF) has been widely used in many scientific fields, such as computer vision, data mining, and bioinformatics. However, NMF is an unsupervised method that cannot fully utilize the label information of data. Thus, its performance is limited in some recognition and classification problems. To remedy this shortcoming, this paper proposes a Semisupervised Discriminative NMF (SDNMF) method. First, we design a Soft-Labeled NMF (SLNMF) model by introducing a soft-label matrix-based regression term into the original NMF, so that the relationship between the soft-label matrix and low-dimensional features can be constructed to improve the discriminative ability of low-dimensional features. Second, to effectively estimate the soft-label matrix, a Label Propagation (LP) model is adopted to fully explore the spatial distribution relationship between the labeled and unlabeled samples. Third, an Adaptive Graph Learning (AGL) model is proposed to exploit the geometric relationship of samples well, which could enhance the performance of LP. Finally, the above three models (i.e., SLNMF, LP, and AGL) are integrated into a unified framework for effective feature learning, which can not only effectively explore the structural relationship matrix between data, but also predict the labels for unknown samples. Moreover, an iterative optimization algorithm is presented to solve our objective function. The convergence and computational complexity analysis of the proposed SDNMF method are also provided. Extensive experiments are conducted on several standard data sets. Compared with related methods, the experimental results verify that the proposed SDNMF method achieves better performance.

A novel regularized asymmetric non-negative matrix factorization for text clustering

Non-negative matrix factorization (NMF) is a dimension reduction method that extracts semantic features from high-dimensional data. Most of the developed optimization methods for NMF only pay attention to how each feature vector of factorized matrices should be modeled, and ignore the relationships among feature vectors. Such a relationship among documents’ feature vectors provides better factorization for text clustering. This paper proposes a novel regularized asymmetric non-negative matrix factorization (RANMF) for text clustering. The proposed method puts regularized constraints on pairwise feature vectors by applying penalties using distance-based measures. We design a new cost function based on the Kullback–Leibler divergence and develop an optimization scheme to solve the cost function by suggesting novel multiplicative updating rules. The proposed method considers the documents from the same cluster closely together in the new representation space. Hence, the acquired parts-based representation has consistent cluster labeling with the original space and has a more discriminating ability. The complexity analysis showed that RANMF does not increase time cost by applying regularizers when comparing with the original NMF. Regarding experiments, the proposed RANMF converges very fast because it terminates in less than ten iterations. The complete proof of convergence and experimental results on the benchmark data sets demonstrate that the proposed multiplicative updating rules converge fast and achieve superior results compared to other algorithms.

Non-negative matrix factorization via adaptive sparse graph regularization

Non-negative matrix factorization (NMF), as an efficient and intuitive dimension reduction algorithm, has been successfully applied to clustering tasks. However, there are still two dominating limitations. First, the original NMF only pays attention to the global data structure, ignoring the intrinsic geometry of the original higher-dimensional data. Second, the traditional pairwise distance-based graph construction is sensitive to noise and outliers, and the nearest neighbor graph obtained is not optimal. As a result, the clustering performance will be reduced. To solve the aforementioned problems and increase the cluster accuracy, a non-negative matrix factorization via adaptive sparse graph (NMF_ASGR) is proposed in this paper. More precisely, this paper assembles the sparse representation and manifold learning into a framework to get the l1 sparse robust graph. The l1 sparse robust graph not only can adaptively discover the potential manifold structure of the data, but also has strong robustness to noise and outliers, which makes the structure of the graph can be learned automatically in the process of matrix decomposition. Moreover, an adaptive sparse graph is learned to batter regularize the NMF. Finally, the effectiveness and superiority of the proposed algorithm are illustrated by lots of image clustering experiments.

A Provable Splitting Approach for Symmetric Nonnegative Matrix Factorization

The symmetric Nonnegative Matrix Factorization (NMF), a special but important class of the general NMF, has found numerous applications in data analysis such as various clustering tasks. Unfortunately, designing fast algorithms for the symmetric NMF is not as easy as for its nonsymmetric counterpart, since the latter admits the splitting property that allows state-of-the-art alternating-type algorithms. To overcome this issue, we first split the decision variable and transform the symmetric NMF to a penalized nonsymmetric one, paving the way for designing efficient alternating-type algorithms. We then show that solving the penalized nonsymmetric reformulation returns a solution to the original symmetric NMF. Moreover, we design a family of alternating-type algorithms and show that they all admit strong convergence guarantee: the generated sequence of iterates is convergent and converges at least sublinearly to a critical point of the original symmetric NMF. Finally, we conduct experiments on both synthetic data and real image clustering to support our theoretical results and demonstrate the performance of the alternating-type algorithms.

A robust semi-supervised NMF model for single cell RNA-seq data.

BackgroundSingle-cell RNA-sequencing (scRNA-seq) technology is a powerful tool to study organism from a single cell perspective and explore the heterogeneity between cells. Clustering is a fundamental step in scRNA-seq data analysis and it is the key to understand cell function and constitutes the basis of other advanced analysis. Nonnegative Matrix Factorization (NMF) has been widely used in clustering analysis of transcriptome data and achieved good performance. However, the existing NMF model is unsupervised and ignores known gene functions in the process of clustering. Knowledges of cell markers genes (genes that only express in specific cells) in human and model organisms have been accumulated a lot, such as the Molecular Signatures Database (MSigDB), which can be used as prior information in the clustering analysis of scRNA-seq data. Because the same kind of cells is likely to have similar biological functions and specific gene expression patterns, the marker genes of cells can be utilized as prior knowledge in the clustering analysis.MethodsWe propose a robust and semi-supervised NMF (rssNMF) model, which introduces a new variable to absorb noises of data and incorporates marker genes as prior information into a graph regularization term. We use rssNMF to solve the clustering problem of scRNA-seq data.ResultsTwelve scRNA-seq datasets with true labels are used to test the model performance and the results illustrate that our model outperforms original NMF and other common methods such as KMeans and Hierarchical Clustering. Biological significance analysis shows that rssNMF can identify key subclasses and latent biological processes. To our knowledge, this study is the first method that incorporates prior knowledge into the clustering analysis of scRNA-seq data.

Elastic net regularized kernel non-negative matrix factorization algorithm for clustering guided image representation

Due to the capacity of data representation, non-negative matrix factorization has been investigated widely by introducing a variety of constraints. The general processing of non-negative matrix factorization for image clustering consists of two steps: (i) achieving the r-dimensional non-negative image representations, where the rank r is set to the expected number of clusters; (ii) adopting the traditional clustering techniques to accomplish the clustering task. Nevertheless, the previous non-negative matrix factorization variants derive image representations from the original space which cannot handle the nonlinear structure of images. This paper focuses on the existing issues and proposes an elastic net regularized kernel non-negative matrix factorization algorithm for clustering guided image representation. In order to explore the nonlinear relations of images, this paper uses the kernel trick to extend original non-negative matrix factorization. A self-organized graph and elastic net regularization are incorporated into the proposed objective of kernel non-negative matrix factorization, besides, the rank is allowed to be larger than the expected number of clusters. By doing so, the graph defined in the feature space is more qualified to represent the intrinsic structure of images. As an accompanying advantage, the clusters of images can be determined using the graph directly without using the two-step trick. According to the proposed alternating update algorithm for solving the optimization problem, the image representation and clustering result can be obtained simultaneously. Extensive experiments on challenging data sets demonstrate the effectiveness of the proposed algorithm compared with the prominent non-negative matrix factorization variants for image clustering.

Mapping Benthic Habitats by Extending Non-Negative Matrix Factorization to Address the Water Column and Seabed Adjacency Effects

Monitoring of coastal areas by remote sensing is an important issue. The interest of using an unmixing method to determine the seabed composition from hyperspectral aerial images of coastal areas is investigated. Unmixing provides both seabed abundances and endmember reflectances. A sub-surface mixing model is presented, based on a recently proposed oceanic radiative transfer model that accounts for seabed adjacency effects in the water column. Two original non-negative matrix factorization ( N M F )-based unmixing algorithms, referred to as W A D J U M (Water ADJacency UnMixing) and W U M (Water UnMixing, no adjacency effects) are developed, assuming as known the water column bio-optical properties. Simulations show that W A D J U M algorithm achieves performance close to that of the N M F -based unmixing of the seabed without any water column, up to 10 m depth. W U M performance is lower and decreases with the depth. The robustness of the algorithms when using erroneous information about the water column bio-optical properties is evaluated. The results show that the abundance estimation is more reliable using W A D J U M approach. W A D J U M is applied to real data acquired along the French coast; the derived abundance maps of the benthic habitats are discussed and compared to the maps obtained using a fixed spectral library and a least-square ( L S ) estimation of the seabed mixing coefficients. The results show the relevance of the W A D J U M algorithm for the local analysis of the benthic habitats.

Robust nonnegative matrix factorization with structure regularization

Nonnegative matrix factorization (NMF) has attracted more and more attention due to its wide applications in computer vision, information retrieval, and machine learning. In contrast to the original NMF and its variants, this paper proposes a novel unsupervised learning framework, called robust structured nonnegative matrix factorization (RSNMF) which respects both global and local structures of the data space. Specifically, to learn a discriminative representation, RSNMF explores both the global structure via considering the data variance and the local structure via exploiting the data neighborhood. To well address the problem of noise and outliers, it imposes joint L2,1-norm minimization on both the loss function of NMF and the regularization of the basis matrix. The geometric structure and the joint L2,1-norm are formulated as an optimization model, which is solved by the proposed iterative algorithm. Finally, the convergence of RSNMF is analyzed theoretically and empirically. The experimental results on real-world data sets show the effectiveness of our proposed algorithm in comparison to state-of-the-art algorithms.

Robust non-negative matrix factorization with multiple correntropy-induced hypergraph regularizer

Non-negative matrix factorization (NMF) is a popular learning tool, which has widely used in computer vision and image processing. Many variants and extensions of NMF have been proposed, where the manifold regularized NMF methods have achieved promising performance due to the preservation of the geometric structures of the data. However, for many applications, the data is usually contaminated by complex noise. The noise leads the data to deviate from the intrinsic manifold, resulting in the degenerate performance. To make the NMF methods reflect the underlying manifold structure well, we propose a novel robust non-negative matrix factorization model. Our model proposes a novel ensemble manifold regularizer, which combines multiple robust hypergraphs to estimate the underlying manifold. Specifically, the correntropy-induced hypergraph is used as the initial manifold estimation. By incorporating the proposed regularizer into the original NMF framework, two novel manifold regularized NMF methods are proposed. The clustering results on the noisy image datasets demonstrate that our model is effective, which achieves the state-of-the-art performance.

Sparse Graph Regularization Non-Negative Matrix Factorization Based on Huber Loss Model for Cancer Data Analysis.

Non-negative matrix factorization (NMF) is a matrix decomposition method based on the square loss function. To exploit cancer information, cancer gene expression data often uses the NMF method to reduce dimensionality. Gene expression data usually have some noise and outliers, while the original NMF loss function is very sensitive to non-Gaussian noise. To improve the robustness and clustering performance of the algorithm, we propose a sparse graph regularization NMF based on Huber loss model for cancer data analysis (Huber-SGNMF). Huber loss is a function between L 1-norm and L 2-norm that can effectively handle non-Gaussian noise and outliers. Taking into account the sparsity matrix and data geometry information, sparse penalty and graph regularization terms are introduced into the model to enhance matrix sparsity and capture data manifold structure. Before the experiment, we first analyzed the robustness of Huber-SGNMF and other models. Experiments on The Cancer Genome Atlas (TCGA) data have shown that Huber-SGNMF performs better than other most advanced methods in sample clustering and differentially expressed gene selection.

Autonomous Semantic Community Detection via Adaptively Weighted Low-rank Approximation

Identification of semantic community structures is important for understanding the interactions and sentiments of different groups of people and predicting the social emotion. A robust community detection method needs to autonomously determine the number of communities and community structure for a given network. Nonnegative matrix factorization (NMF), a component decomposition approach for latent sentiment discovery, has been extensively used for community detection. However, the existing NMF-based methods require the number of communities to be determined a priori , limiting their applicability in practice of affective computing. Here, we develop a novel NMF-based method to autonomously determine the number of semantic communities and community structure simultaneously. In our method, we use an initial number of semantic communities, larger than the actual number, in the NMF formulation, and then suppress some of the communities by introducing an adaptively weighted group-sparse low-rank regularization to derive the target number of communities and at the same time the corresponding community structure. Our method not only maintains the efficiency without increasing the complexity compared to the original NMF method but also can be straightforwardly extended to handle the non-network data. We thoroughly examine the new method, showing its superior performance over several competing methods on synthetic and large real-world social networks.

Another Robust NMF: Rethinking the Hyperbolic Tangent Function and Locality Constraint

Non-negative matrix factorization (NMF) is a classical data analysis tool for clustering tasks. It usually considers the squared loss to measure the reconstruction error, thus it is sensitive to the presence of outliers. Looking into the literature, most of the existing robust NMF models focus on statistics-based robust estimators with known distribution assumptions. Besides those estimators, whether can we seek another function without the distribution assumption to boost the robustness of NMF? To solve the problem, we propose a robust NMF termed as tanh NMF for short, which rethinks the hyperbolic tangent (tanh) function as a robust loss to evaluate the reconstruction error. Moreover, to capture geometric structure within the data, we devise a locality constraint to regularize tanh NMF to model data locality. Owing to the non-convex tanh function, it is non-trivial to optimize tanh NMF. Following the paradigm of the half-quadratic algorithm, we easily solve an adaptive weighted NMF instead of original tanh NMF. The experiments of face clustering on four popular facial datasets with/without corruptions show that the proposed method achieves the satisfactory performance against several representative baselines including NMF and its robust counterparts. This also implies that the proposed tanh function could serve as an alternative robust loss for NMF.

Robust Adaptive Graph Regularized Non-Negative Matrix Factorization

Data clustering, which aims to divide the given samples into several different groups, has drawn much attention in recent years. As a powerful tool, non-negative matrix factorization (NMF) has been applied successfully in clustering tasks. However, there are still two main limitations. First, the original NMF treats equally both noisy and clean data, which leads to high sensitivity to noises and outliers. Second, the performance of graph-based NMFs highly depends on the input graph, that is, if a low-quality graph is constructed to regularize NMF, the clustering results will be bad. To address the above-mentioned problems, we propose a novel robust adaptive graph regularized non-negative matrix factorization (RAGNMF) for data clustering. To be specific, we develop a robust weighted NMF (RWNMF) that can assign small weights to noises and outliers and large weights to clean data. Thus, the robustness of NMF is improved. Moreover, in the process of matrix factorization, metric learning is combined to choose some discriminative features and compute more appropriate distances of samples. Then, an adaptive graph is learned to well regularize the NMF. The experimental results demonstrate that the proposed RAGNMF can achieve better clustering performance then most of the state-of-the-art methods.

Rank-Adaptive Non-Negative Matrix Factorization

Dimension reduction is a challenge task in data processing, especially in high-dimensional data processing area. Non-negative matrix factorization (NMF), as a classical dimension reduction method, has a contribution to the parts-based representation for the characteristics of non-negative constraints in the NMF algorithm. In this paper, the NMF algorithm is introduced to extract local features for dimension reduction. Considering the problem of which NMF is required to define the number of the decomposition rank manually, we proposed a rank-adaptive NMF algorithm, in which the affinity propagation (AP) clustering algorithm is introduced to determine adaptively the number of the decomposition rank of NMF. Then, the rank-adaptive NMF algorithm is used to extract features for the original image. After that, a low-dimensional representation of the original image is obtained through the projection from the original images to the feature space. Finally, we used extreme learning machine (ELM) and k-nearest neighbor (KNN) as the classifier to classify those low-dimensional feature representations. The experimental results demonstrate that the decomposition rank determined by the AP clustering algorithm can reflect the characteristics of the original data. When it is combined with the classification algorithm ELM or KNN and applied to handwritten character recognition, the proposed method not only reduces the dimension of original images but also performs well in terms of classification accuracy and time consumption. A new rank-adaptive NMF algorithm is proposed based on the AP clustering algorithm and the original NMF algorithm. According to this algorithm, the low-dimensional representation of the original data can be obtained without any prior knowledge. In addition, the proposed rank-adaptive NMF algorithm combined with the ELM and KNN classification algorithms performs well.

Label and orthogonality regularized non-negative matrix factorization for image classification

As one of the most popular data-representation methods, non-negative matrix factorization (NMF) has been widely used in image processing and pattern recognition. Compared with other dimension reduction methods, we can interpret the data with psychological intuition using NMF since NMF can decompose the whole into visual parts by learning the non-negative basis. However, the original NMF lacks of extracting the discriminant information of the data for the image classification task. For enhancing the discriminant and parts-based interpretability, this work proposes a label and orthogonality regularized NMF (LONMF) algorithm based on the squared Euclidean distance. LONMF takes into account the label consistence with the low-dimensional projected data and orthogonal property of the non-negative basis. By integrating the non-negative constraint, label consistence, and orthogonal property into the objective function, the efficient updating procedure can obtain a discriminant basis matrix. Meanwhile, we design a linear classifier using the projected data to guide the label for efficient image classification task. Experiment results of the competitive NMF variants on the challenging digit and face databases demonstrate the effectiveness of the proposed LONMF algorithm.

Joint Linear Regression and Nonnegative Matrix Factorization Based on Self-Organized Graph for Image Clustering and Classification

Nonnegative matrix factorization (NMF) technique has been developed successfully to represent the intuitively meaningful feature of data. A suitable representation can faithfully preserve the intrinsic structure of data. Due to the fact that it introduces the label information, semi-supervised NMF has been demonstrated more advantageous in image representation than original NMF. However, previous semi-supervised NMF variants construct a label indicator matrix only for tagging the labeled data and not being optimized together with the matrix factorization. It is short of label propagation and fails to work for predicting the attribution of data. Moreover, the transductive semi-supervised NMF variants cannot dispose the prediction of unseen data, restricting the application of NMF. In this paper, a joint optimization framework of linear regression and NMF (LR-NMF) based on the self-organized graph is proposed for a completed task which simultaneously takes into account image representation and attribution prediction. By minimizing the proposed objective, three interactive threads are running: decomposing the data into nonnegative basis matrix and the corresponding representation, linear regression using the nonnegative representation, and label propagation based on the self-organized graph which is defined in the feature space. The products of LR-NMF can be viewed as extracting nonnegative feature for clustering, meanwhile, they can be used to solve the out-of-sample problem for classification. Extensive clustering and classification experiments on the digit, face, and object challenging data sets are presented to show the efficacy of the proposed LR-NMF algorithm.

Nonnegative Discriminant Matrix Factorization

Nonnegative matrix factorization (NMF), which aims at obtaining the nonnegative low-dimensional representation of data, has received wide attention. To obtain more effective nonnegative discriminant bases from the original NMF, in this paper, a novel method called nonnegative discriminant matrix factorization (NDMF) is proposed for image classification. NDMF integrates the nonnegative constraint, orthogonality, and discriminant information in the objective function. NDMF considers the incoherent information of both factors in standard NMF and is proposed to enhance the discriminant ability of the learned base matrix. NDMF projects the low-dimensional representation of the subspace of the base matrix to regularize the NMF for discriminant subspace learning. Based on the Euclidean distance metric and the generalized Kullback-Leibler (KL) divergence, two kinds of iterative algorithms are presented to solve the optimization problem. The between- and within-class scatter matrices are divided into positive and negative parts for the update rules and the proofs of the convergence are also presented. Extensive experimental results demonstrate the effectiveness of the proposed method in comparison with the state-of-the-art discriminant NMF algorithms.

Discriminative non-negative matrix factorization (DNMF) and its application to the fault diagnosis of diesel engine

Diesel engines, widely used in engineering, are very important for the running of equipments and their fault diagnosis have attracted much attention. In the past several decades, the image based fault diagnosis methods have provided efficient ways for the diesel engine fault diagnosis. By introducing the class information into the traditional non-negative matrix factorization (NMF), an improved NMF algorithm named as discriminative NMF (DNMF) was developed and a novel imaged based fault diagnosis method was proposed by the combination of the DNMF and the KNN classifier. Experiments performed on the fault diagnosis of diesel engine were used to validate the efficacy of the proposed method. It is shown that the fault conditions of diesel engine can be efficiently classified by the proposed method using the coefficient matrix obtained by DNMF. Compared with the original NMF (ONMF) and principle component analysis (PCA), the DNMF can represent the class information more efficiently because the class characters of basis matrices obtained by the DNMF are more visible than those in the basis matrices obtained by the ONMF and PCA.

Projective robust nonnegative factorization

Nonnegative matrix factorization (NMF) has been successfully used in many fields as a low-dimensional representation method. Projective nonnegative matrix factorization (PNMF) is a variant of NMF that was proposed to learn a subspace for feature extraction. However, both original NMF and PNMF are sensitive to noise and are unsuitable for feature extraction if data is grossly corrupted. In order to improve the robustness of NMF, a framework named Projective Robust Nonnegative Factorization (PRNF) is proposed in this paper for robust image feature extraction and classification. Since learned projections can weaken noise disturbances, PRNF is more suitable for classification and feature extraction. In order to preserve the geometrical structure of original data, PRNF introduces a graph regularization term which encodes geometrical structure. In the PRNF framework, three algorithms are proposed that add a sparsity constraint on the noise matrix based on L1/2 norm, L1 norm, and L2, 1 norm, respectively. Robustness and classification performance of the three proposed algorithms are verified with experiments on four face image databases and results are compared with state-of-the-art robust NMF-based algorithms. Experimental results demonstrate the robustness and effectiveness of the algorithms for image classification and feature extraction.