Nonnegative Matrix Factorization Model Research Articles

Minimum Volume Constraint with Perturbation for Non-negative Matrix Factorization

Nonnegative matrix factorization (NMF) has been applied in hyperspectral unmixing. The nonconvexity of the NMF’s cost function leads to solutions that are only locally optimal. Adding regularized terms to the NMF helps improve the solutions. In this study, we proposed a regularized NMF model, the regularized tern is the minimum volume constraint with perturbation. The NMF model is solved with multiplicative updated rules. Numerical results verified that adding a disturbance term to the minimum volume constraint effectively improves the spectral curve’s local accuracy while maintaining the original model’s advantages.

Class-driven nonnegative matrix factorization with manifold regularization for data clustering

Nonnegative matrix factorization (NMF) is an effective technique to extract the underlying low-dimensional structure of data by utilizing its parts-based representation, which has been widely used in feature extraction and machine learning. However, NMF is an unsupervised learning algorithm without utilizing the discriminative prior information. In this paper, we put forward a new class-driven NMF with manifold regularization (MCDNMF) algorithm, which incorporates both the local manifold regularization and the label information of data into the NMF model. Specifically, MCDNMF not only encodes the local geometrical structure of data space by using the manifold regularization, but also takes the available label information by introducing the class-driven constraint. This class-driven constraint forces the new representations of data points to be more similar within the same class while different between other classes. Therefore, the discriminative abilities of clustering are greatly boosted. Experimental results on several datasets validate the effectiveness of proposed MCDNMF in comparison with the other state-of-the-art methods.

Cluster structure augmented deep nonnegative matrix factorization with low-rank tensor learning

Clustering approaches based on deep nonnegative matrix factorization have received significant attention because it can learn hierarchical semantics from data in a layer-wise manner. However, latent representation learning and clustering are two separate processes in these models, leading to information loss and sub-optimal clusterings. Furthermore, if data contain complex structure or noise, the pre-defined graph in data space will not be precise enough for graph-regularized deep nonnegative matrix factorization models. Moreover, most of them maintain the similarity graph at the top layer, neglecting the information covered by other layers. In this paper, Cluster Structure Augmented Deep Nonnegative Matrix Factorization with Low-rank Tensor Learning is proposed. First, as representations at different layers cover different data abstractions, a learning mechanism is leveraged to generate multiple local similarity graphs based on representations from different layers, thus maintaining diversity across layers. Then, a low-rank third-order tensor is constructed by stacking these graphs to capture the high-order consistency across layers, thus capturing the intrinsic property of data. Third, clustering is integrated into the framework, allowing the cluster structure to facilitate multi-layer matrix factorization and graph structure learning, thereby generating discriminative representations. Experimental results on several datasets demonstrate that the proposed model outperforms several state-of-the-art methods.

Community Detection in Multiplex Networks Using Orthogonal Non-Negative Matrix Tri-Factorization Based on Graph Regularization and Diversity

In recent years, community detection has received increasing interest. In network analysis, community detection refers to the identification of tightly connected subsets of nodes, which are called “communities” or “groups”, in the network. Non-negative matrix factorization models are often used to solve the problem. Orthogonal non-negative matrix tri-factorization (ONMTF) exhibits significant potential as an approach for community detection within multiplex networks. This paper explores the application of ONMTF in multiplex networks, aiming to detect both shared and exclusive communities simultaneously. The model decomposes each layer within the multiplex network into two low-rank matrices. One matrix corresponds to shared communities across all layers, and the other to unique communities within each layer. Additionally, graph regularization and the diversity of private communities are taken into account in the algorithm. The Hilbert Schmidt Independence Criterion (HSIC) is used to constrain the independence of private communities. The results prove that ONMTF effectively addresses community detection in multiplex networks. It also offers strong interpretability and feature extraction capabilities. Therefore, it is an advanced method for community detection in multiplex networks.

Symmetry and Graph Bi-Regularized Non-Negative Matrix Factorization for Precise Community Detection

Community is a fundamental and highly desired pattern in a Large-scale Undirected Network (LUN). Community detection is a vital issue when LUN representation learning is performed. Owing to its good scalability and interpretability, a Symmetric and Non-negative Matrix Factorization model is frequently utilized to tackle this issue. It adopts a unique Latent Factor (LF) matrix for precisely representing LUN’s symmetry, which, unfortunately, leads to a reduced LF space that decreases its representation learning ability to a target LUN. Motivated by this discovery, this study proposes a Symmetry and Graph Bi-regularized Non-negative Matrix Factorization (B-NMF) method that: a) leverages multiple LF matrices when representing LUN, thereby boosting the representation learning ability; b) constructs a symmetry regularization term that implies the equality constraint among its multiple LF matrices, thereby illustrating LUN’s intrinsic symmetry; and c) incorporates graph regularization into its learning objective, thereby illustrating LUN’s local geometry. A theoretical proof is given to theoretically validate B-NMF’s convergence ability. The regularization hyperparameters are selected by validating model modularity, thereby guaranteeing B-NMF’s practicability in addressing real application issues. Extensive experimental results on ten LUNs from real applications demonstrate that the proposed B-NMF-based community detector significantly outperforms several baseline and state-of-the-art models in achieving highly-accurate community detection results. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —LUNs are very-commonly seen in real applications like a social network system. Communities in LUNs are vital for various knowledge discovery-related applications. For accurately detecting them, a detector should guarantee its high representation learning ability to a target LUN. To do so, this paper presents a B-NMF model that is able to perform precise representation learning to LUNs, thereby achieving accurate community detection results. In comparison with conventional Symmetric and Non-negative Matrix Factorization-based community detectors, a B-NMF-based community detector enjoys its enlarged latent feature space, which ensures its higher representation ability to a target LUN. It depends on two regularization hyperparameters, which can be selected by performing grid-search on the target LUN via its modularity evaluation. This paper gives the empirical values of B-NMF’s regularization hyperparameters based on the parametersensitivity tests on the involved experimental datasets. The proposed B-NMF model is shown to be highly suitable for addressing community detection and clustering tasks on LUNs from real applications.

Orthogonal graph regularized non-negative matrix factorization under sparse constraints for clustering

The standard NMF algorithm is not suitable for sampling data from low-dimensional manifolds embedded in high-dimensional environmental spaces, as the geometric information hidden in feature manifolds and sample manifolds is rarely learned. In order to obtain better clustering performance based on NMF, manifold and orthogonal constraint, a new type of model named Orthogonal Graph regularized Non-negative Matrix Factorization model under Sparse Constraints (OGNMFSC) is proposed. Firstly, this type of model constructs a nearest neighbor graph to encode the geometric information of the data space, in order to obtain more discriminative ability by preserving the structure of the graph. Secondly, this type of model adds orthogonal constraints to achieve better local representation and significantly reduce the inconsistency between the original matrix and the basis vectors. Thirdly, by adding sparse constraints to obtain a sparser representation matrix, the clustering performance of the model can be improved. The main conclusion of this paper is that two effective algorithms have been generated to solve the model, which not only provides theoretical convergence proof for these two algorithms, but also demonstrates significant clustering performance in experiments compared to classical models such as K-means, PCA, NMF, Semi-NMF, NMFSC, ONMF, GNMF, NeNMF.

Dual auto-weighted multi-view clustering via autoencoder-like nonnegative matrix factorization

Multi-view clustering (MVC) can exploit the complementary information among multi-view data to achieve the satisfactory performance, thus having extensive potentials for practical applications. Although Nonnegative Matrix Factorization (NMF) has emerged as an effective technique for MVC, the existing NMF-based methods still have two main limitations: 1) They solely focus on the reconstruction of original data, which can be regarded as the decoder of an autoencoder, while neglecting the low-dimensional representation learning. 2) They lack the ability to effectively capture both linear and nonlinear structures of data. To solve these problems, in this paper, we propose a Dual Auto-weighted multi-view clustering model based on Autoencoder-like NMF (DA2NMF), which enables a comprehensive exploration of both linear and nonlinear structures. Specifically, we establish an autoencoder-like NMF model that learns linear low-dimensional representations by integrating data reconstruction and representation learning within a unified framework. Moreover, the adaptive graph learning is introduced to explore the nonlinear structures in data. We further design a dual auto-weighted strategy to adaptively compute weights for different views and low-dimensional representations, thereby obtaining an enhanced consistent graph. An effective algorithm based on Multiplicative Update Rule (MUR) is developed to solve the DA2NMF with the theoretical convergence guarantee. Experimental results show that the proposed DA2NMF can effectively improve the clustering performance compared with the state-of-the-art MVC algorithms.

Singular Value Decomposition-Driven Non-negative Matrix Factorization with Application to Identify the Association Patterns of Sarcoma Recurrence.

Sarcomas are malignant tumors from mesenchymal tissue and are characterized by their complexity and diversity. The high recurrence rate making it important to understand the mechanisms behind their recurrence and to develop personalized treatments and drugs. However, previous studies on the association patterns of multi-modal data on sarcoma recurrence have overlooked the fact that genes do not act independently, but rather function within signaling pathways. Therefore, this study collected 290 whole solid images, 869 gene and 1387 pathway data of over 260 sarcoma samples from UCSC and TCGA to identify the association patterns of gene-pathway-cell related to sarcoma recurrences. Meanwhile, considering that most multi-modal data fusion methods based on the joint non-negative matrix factorization (NMF) model led to poor experimental repeatability due to random initialization of factorization parameters, the study proposed the singular value decomposition (SVD)-driven joint NMF model by applying the SVD method to calculate initialized weight and coefficient matrices to achieve the reproducibility of the results. The results of the experimental comparison indicated that the SVD algorithm enhances the performance of the joint NMF algorithm. Furthermore, the representative module indicated a significant relationship between genes in pathways and image features. Multi-level analysis provided valuable insights into the connections between biological processes, cellular features, and sarcoma recurrence. In addition, potential biomarkers were uncovered, while various mechanisms of sarcoma recurrence were identified from an imaging genetic perspective. Overall, the SVD-NMF model affords a novel perspective on combining multi-omics data to explore the association related to sarcoma recurrence.

Topic modeling and social network analysis approach to explore diabetes discourse on Twitter in India.

The utilization of social media presents a promising avenue for the prevention and management of diabetes. To effectively cater to the diabetes-related knowledge, support, and intervention needs of the community, it is imperative to attain a deeper understanding of the extent and content of discussions pertaining to this health issue. This study aims to assess and compare various topic modeling techniques to determine the most effective model for identifying the core themes in diabetes-related tweets, the sources responsible for disseminating this information, the reach of these themes, and the influential individuals within the Twitter community in India. Twitter messages from India, dated between 7 November 2022 and 28 February 2023, were collected using the Twitter API. The unsupervised machine learning topic models, namely, Latent Dirichlet Allocation (LDA), non-negative matrix factorization (NMF), BERTopic, and Top2Vec, were compared, and the best-performing model was used to identify common diabetes-related topics. Influential users were identified through social network analysis. The NMF model outperformed the LDA model, whereas BERTopic performed better than Top2Vec. Diabetes-related conversations revolved around eight topics, namely, promotion, management, drug and personal story, consequences, risk factors and research, raising awareness and providing support, diet, and opinion and lifestyle changes. The influential nodes identified were mainly health professionals and healthcare organizations. The study identified important topics of discussion along with health professionals and healthcare organizations involved in sharing diabetes-related information with the public. Collaborations among influential healthcare organizations, health professionals, and the government can foster awareness and prevent noncommunicable diseases.

Speech Enhancement Using Joint DNN-NMF Model Learned with Multi-Objective Frequency Differential Spectrum Loss Function

We propose a multi-objective joint model of non-negative matrix factorization (NMF) and deep neural network (DNN) with a new loss function for speech enhancement. The proposed loss function (LMOFD) is a weighted combination of a frequency differential spectrum mean squared error (MSE)-based loss function (LFD) and a multi-objective MSE loss function LMO. The conventional MSE loss function computes the discrepancy between the estimated speech and clean speech across all frequencies, disregarding the process of changing amplitude in the frequency domain which contains valuable information. The differential spectrum representation retains spectral peaks that carry important information. Using this representation helps to ensure that this information in the speech signal is reserved. Also, on the other hand, noise spectra typically have a flat shape and as the differential operation makes the flat spectral partly close to zero, the differential spectrum is resistant to noises with smooth structures. Thus, we propose using a frequency-differentiated loss function that considers the magnitude spectrum differentiations between the neighboring frequency bins in each time frame. This approach maintains the spectrum variations of the objective signal in the frequency domain, which can effectively reduce the noise deterioration effects. The multi-objective MSE term LMO is a combined two-loss function related to the NMF coefficients which are the intermediate output targets, and the original spectral signals as the actual output targets. The use of encoded NMF coefficients as low-dimensional structural features for DNN serves as prior knowledge and helps the learning process. LMO is used beside LFD to take advantage of both the properties of the original and the differential spectrum in the training loss function. Moreover, a DNN-based noise classification and fusion strategy (NCF) is proposed to exploit a discriminative model for noise reduction. The experiments reveal the improvements of the proposed approach compared to the previous methods.

Resolving Ultraviolet-Visible Spectra for Complex Dissolved Mixtures of Multitudinous Organic Matters in Aerosols.

Light-absorbing organic aerosols, referred to as brown carbon (BrC), play a vital role in the global climate and air quality. Due to the complexity of BrC chromophores, the identified absorbing substances in the ambient atmosphere are very limited. However, without comprehensive knowledge of the complex absorbing compounds in BrC, our understanding of its sources, formation, and evolution mechanisms remains superficial, leading to great uncertainty in climatic and atmospheric models. To address this gap, we developed a constrained non-negative matrix factorization (NMF) model to resolve the individual ultraviolet-visible spectrum for each substance in dissolved organic aerosols, with the power of ultrahigh-performance liquid chromatography-diode array detector-ultrahigh-resolution mass spectrometry (UHPLC-DAD-UHRMS). The resolved spectra were validated by selected standard substances and validation samples. Approximately 40,000 light-absorbing substances were recognized at the MS1 level. It turns out that BrC is composed of a vast number of substances rather than a few prominent chromophores in the urban atmosphere. Previous understanding of the absorbing feature of BrC based on a few identified compounds could be biased. Weak-absorbing substances missed previously play an important role in BrC absorption when they are integrated due to their overwhelming number. This model brings the property exploration of complex dissolved organic mixtures to a molecular level, laying a foundation for identifying potentially significant compositions and obtaining a comprehensive chemical picture.

Learning Single Spectral Abundance for Hyperspectral Subpixel Target Detection.

Due to the limitation of target size and spatial resolution, targets of interest in hyperspectral images (HSIs) often appear as subpixel targets, which makes hyperspectral target detection still faces an important bottleneck, that is, subpixel target detection. In this article, we propose a new detector by learning single spectral abundance for hyperspectral subpixel target detection (denoted as LSSA). Different from most existing hyperspectral detectors that are designed based on a match of the spectrum assisted by spatial information or focusing on the background, the proposed LSSA addresses the problem of detecting subpixel targets by learning a spectral abundance of the target of interest directly. In LSSA, the abundance of the prior target spectrum is updated and learned, while the prior target spectrum is fixed in a nonnegative matrix factorization (NMF) model. It turns out that such a way is quite effective to learn the abundance of subpixel targets and contributes to detecting subpixel targets in hyperspectral imagery (HSI). Numerous experiments are conducted on one simulated dataset and five real datasets, and the results indicate that the LSSA yields superior performance in hyperspectral subpixel target detection and outperforms its counterparts.

Inter- and intra-hypergraph regularized nonnegative matrix factorization with hybrid constraints

The accurate low rank representation of high-dimensional data learned by the manifold regularized nonnegative matrix factorization framework is effective in data clustering. In previous work, this work has mainly been solved in the way of similarity matrix induction. To further increase the efficacy of low-rank representations, we propose a novel semi-supervised non-negative matrix factorization (NMF) model in this study called inter- and intra-hypergraph regularized non-negative matrix factorization with hybrid constraints (IGNMFC). IGNMFC constructs intra-hypergraph regularization and intra-hypergraph regularization by hypergraph learning, which can precisely induce high-dimensional data to map toward low-dimensional. Moreover, hybrid constraints are introduced to improve the exclusivity and sparsity of low-dimensional representations, and the result accounts for the benefit of this method in learning distinguishable subspace representations. Finally, IGNMFC is transformed into an optimal problem and an efficient iteration rule is proposed. Experiments on several datasets demonstrate that the proposed method outperforms the state-of-the-art NMF algorithms, and can achieve at least 7.9%∼15% accuracy improvement in most cases.

Incremental semi-supervised graph learning NMF with block-diagonal

Non-negative matrix factorization (NMF), as a good data dimensionality reduction method, is widely used in the field of image recognition. Incremental non-negative matrix factorization (INMF) as an improvement solves the problem of inefficiency caused by repeated running of data samples during online learning. However, in the traditional incremental non-negative matrix factorization algorithm, the newly added training samples do not contain label information. Some samples may be marked in both the initial sample and the new training sample in the real application scenario. In order to make full use of the label information carried by the dataset, in this paper, we propose a semi-supervised non-negative matrix factorization model for batch incremental data, incremental semi-supervised graph learning NMF with block diagonal (ISGDNMF). The model is divided into three cases according to the degree of label-carrying for the new batch data: all label-carrying, no label-carrying, and partial label-carrying. The label information is also used to add a diagonal structure to the coefficient matrix, which makes it possible to have stronger discriminatory ability and to distinguish different classes of images more easily. And graph regularization information is added in order to maintain the spatial-geometric structure of the data. Experiments on six image datasets show that this algorithm has superior performance relative to the other seven NMF-based algorithms.

Joint orthogonal symmetric non-negative matrix factorization for community detection in attribute network

Community detection is an important and challenging task in complex attribute network analysis. Symmetric non-negative matrix factorization-based methods have become promising because of their excellent ability to extract low-dimensional representations of attribute networks (which are characterized by their adjacency and attribute data). In this paper, we propose a novel community detector called joint symmetric non-negative matrix factorization model, in which the attribute homogeneity and topology similarities of an attribute network are characterized in a unified framework. An orthogonal constraint is imposed on the factor matrix to improve the accuracy of nodes’ affiliations to communities. Furthermore, a novel multi-order graph regularization was developed to preserve the intrinsic geometric structure, and an eigenvector centrality-based enhancement strategy was established to explore more comprehensive adjacency information. Extensive experiments on community detection tasks demonstrate that the proposed method performs significantly better than existing state-of-the-art methods in most benchmark complex networks.

A Novel Underdetermined Blind Source Separation Algorithm of Frequency-Hopping Signals via Time-Frequency Analysis

To address the significant performance degradation of conventional underdetermined blind source separation algorithms for frequency-hopping (FH) signals under time-frequency (TF) overlapping conditions, this paper presents a novel three-stage scheme based on the TF distribution of FH signals. In the first stage, key parameters of the FH signal are estimated using a TF binary graph. In the second step, the initial mixing matrix is estimated for non-overlapping and overlapping carrier frequencies employing density peaks clustering and tensor decomposition methods, respectively. In the third step, the final mixing matrix, directions of arrival (DOA), and source signals are estimated using the expectation-maximization algorithm within the nonnegative matrix factorization model. Finally, different segments of the FH signals are spliced together based on the DOAs of different source signals. Comprehensive experimental results demonstrate the superior performance of the proposed algorithm compared to state-of-the-art algorithms. Even at a signal-to-noise ratio of 5 dB, the correlation coefficient of the estimated source signals can still reach 0.91.

Spatial deduction of mining-induced stress redistribution using an optimized non-negative matrix factorization model

Spatial deduction of mining-induced stress redistribution using an optimized non-negative matrix factorization model

Rank selection for non-negative matrix factorization.

Non-Negative Matrix Factorization (NMF) is a widely used dimension reduction method that factorizes a non-negative data matrix into two lower dimensional non-negative matrices: one is the basis or feature matrix which consists of the variables and the other is the coefficients matrix which is the projections of data points to the new basis. The features can be interpreted as sub-structures of the data. The number of sub-structures in the feature matrix is also called the rank. This parameter controls the model complexity and is the only tuning parameter for the NMF model. An appropriate rank will extract the key latent features while minimizing the noise from the original data. However due to the large amount of optimization error always present in the NMF computation, the rank selection has been a difficult problem. We develop a novel rank selection method based on hypothesis testing, using a deconvolved bootstrap distribution to assess the significance level accurately. Through simulations, we compare our method with a rank selection method based on hypothesis testing using bootstrap distribution without deconvolution and a method based on cross-validation; we demonstrate that our method is not only accurate at estimating the true ranks for NMF, especially when the features are hard to distinguish, but also efficient at computation. When applied to real microbiome data (eg, OTU data and functional metagenomic data), our method also shows the ability to extract interpretable subcommunities in the data.

Adaptive neuro-fuzzy inference system model driven by the non-negative matrix factorization-extracted muscle synergy patterns to estimate lower limb joint movements

Background and objectiveFor patients with movement disorders, the main clinical focus is on exercise rehabilitation to help recover lost motor function, which is achieved by relevant assisted equipment. The basis for seamless control of the assisted equipment is to achieve accurate inference of the user's movement intentions in the human-machine interface. This study proposed a novel movement intention detection technology for estimating lower limb joint continuous kinematic variables following muscle synergy patterns, to develop applications for more efficient assisted rehabilitation training. MethodsThis study recruited 16 healthy males and 16 male patients with symptomatic patellar tendinopathy (VISA-P: 59.1 ± 8.7). The surface electromyography of 12 muscles and lower limb joint kinematic and kinetic data from healthy subjects and patients during step-off landings from 30 cm-high stair steps were collected. We subsequently solved the preprocessed data based on the established recursive model of second-order differential equation to obtain the muscle activation matrix, and then imported it into the non-negative matrix factorization model to obtain the muscle synergy matrix. Finally, the lower limb neuromuscular synergy pattern was then imported into the developed adaptive neuro-fuzzy inference system non-linear regression model to estimate the human movement intention during this movement pattern. ResultsSix muscle synergies were determined to construct the muscle synergy pattern driven ANFIS model. Three fuzzy rules were determined in most estimation cases. Combining the results of the four error indicators across the estimated variables indicates that the current model has excellent estimated performance in estimating lower limb joint movement. The estimation errors between the healthy (Angle: R2=0.98±0.03; Torque: R2=0.96±0.04) and patient (Angle: R2=0.98±0.02; Torque: R2=0.96±0.03) groups are consistent. ConclusionThe proposed model of this study can accurately and reliably estimate lower limb joint movements, and the effectiveness will also be radiated to the patient group. This revealed that our models also have certain advantages in the recognition of motor intentions in patients with relevant movement disorders. Future work from this study can be focused on sports rehabilitation in the clinical field by achieving more flexible and precise movement control of the lower limb assisted equipment to help the rehabilitation of patients.

Multiview Clustering via Hypergraph Induced Semi-Supervised Symmetric Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) based multiview technique has been commonly used in multiview data clustering tasks. However, previous NMF based multiview clustering approaches fail to take advantage of a small amount of supervisory information to effectively improve the clustering performance, and are easily affected by the additional post-processing method in clustering tasks. To cope with these issues, a novel framework named multiview clustering via hypergraph induced semi-supervised symmetric NMF (MVCHSS) is proposed in this paper for multiview data clustering applications. Specifically, the proposed method has the following features: 1) a new multiview based hypergraph pairwise constraints propagation (MHPCP) algorithm is developed in MVCHSS to construct a set of informative similarity matrices, revealing the high-order relationships effectively and fully utilizing the limited pairwise constraint supervisory information among samples of each view data; 2) the obtained similarity matrices with much supervisory information are not only enforced into the symmetric NMF (SNMF) model, but also incorporated into the graph regularization for each view data; 3) the optimization problem of MVCHSS is formulated for multiview data clustering tasks to acquire a more discriminative clustering indicator matrix (or called consensus assignment matrix) without additional post-processing method. Moreover, the proof of convergence and the computational complexity for MVCHSS are presented. Extensive experiments on five multiview datasets demonstrate that the proposed MVCHSS framework outperforms several state-of-the-art multiview clustering methods.