Symmetric Nonnegative Matrix Factorization Research Articles

Efficient method for symmetric nonnegative matrix factorization with an approximate augmented Lagrangian scheme

In this paper, we propose an efficient method for solving symmetric nonnegative matrix factorization following an approximate augmented Lagrangian scheme. The augmented Lagrangian subproblem was solved column by column under the block coordinate descent (BCD) framework. In particular, we extend the recursive formula for rank-k nonnegative least squares problems by Chu et al. (2021) to subproblems generated by BCD framework in our method. Thereafter we derive the closed-form solution for k=4. Experiments for clustering show that the proposed method is noticeably efficient and achieves competitive performance compared with state-of-the-art methods.

Hypergraph-based convex semi-supervised unconstraint symmetric matrix factorization for image clustering

Semi-supervised symmetric nonnegative matrix factorization (SNMF) has been extensively utilized in both linear and nonlinear data clustering tasks. However, the current SNMF model's non-convex objective function faces challenges in global optimization and time efficiency. In this study, we leverage label information to propose a convex and unconstrained symmetric matrix factorization (SMF) model that is thoroughly analyzed for its convexity properties. In order to capture high-order relationships among data, a hypergraph is utilized in the model, which is computationally simple, translation invariant, and naturally normalized. Moreover, based on the analysis and the corresponding experiments in the paper, the model exhibits robustness towards outliers to some extent. Due to the convexity of our proposed model without constraint, it can be efficiently optimized using the Conjugate Gradient (CG) method, one of the most efficient methods available. Therefore, we propose a novel Convex Combination-based Sufficient Descent CG (CSDCG) method, which outperforms other methods across 284 optimization problems within the CUTEst library. In order to evaluate the effectiveness of the proposed method, the semi-supervised clustering experiments are conducted on the eight datasets by comparison with ten state-of-the-art matrix factorization (MF) methods. The experiment results demonstrate its superiority over the other compared methods to handle the clustering problem with better performance and less computational time. The code is available at https://github.com/Pokemer/HCSSMF.

Constrained Symmetric Non-Negative Matrix Factorization with Deep Autoencoders for Community Detection

Recently, community detection has emerged as a prominent research area in the analysis of complex network structures. Community detection models based on non-negative matrix factorization (NMF) are shallow and fail to fully discover the internal structure of complex networks. Thus, this article introduces a novel constrained symmetric non-negative matrix factorization with deep autoencoders (CSDNMF) as a solution to this issue. The model possesses the following advantages: (1) By integrating a deep autoencoder to discern the latent attributes bridging the original network and community assignments, it adeptly captures hierarchical information. (2) Introducing a graph regularizer facilitates a thorough comprehension of the community structure inherent within the target network. (3) By integrating a symmetry regularizer, the model’s capacity to learn undirected networks is augmented, thereby facilitating the precise detection of symmetry within the target network. The proposed CSDNMF model exhibits superior performance in community detection when compared to state-of-the-art models, as demonstrated by eight experimental results conducted on real-world networks.

Changes in Community Structure of Brain Dynamic Functional Connectivity States in Mild Cognitive Impairment

Recent researches have noted many changes of short-term dynamic modalities in mild cognitive impairment (MCI) patients' brain functional networks. In this study, the dynamic functional brain networks of 82 MCI patients and 85 individuals in the normal control (NC) group were constructed using the sliding window method and Pearson correlation. The window size was determined using single-scale time-dependent (SSTD) method. Subsequently, k-means was applied to cluster all window samples, identifying three dynamic functional connectivity (DFC) states. Collective sparse symmetric non-negative matrix factorization (cssNMF) was then used to perform community detection on these states and quantify differences in brain regions. Finally, metrics such as within-community connectivity strength, community strength, and node diversity were calculated for further analysis. The results indicated high similarity between the two groups in state 2, with no significant differences in optimal community quantity and functional segregation (p < 0.05). However, for state 1 and state 3, the optimal community quantity was smaller in MCI patients compared to the NC group. In state 1, MCI patients had lower within-community connectivity strength and overall strength than the NC group, whereas state 3 showed results opposite to state 1. Brain regions with statistical difference included MFG.L, ORBinf.R, STG.R, IFGtriang.L, CUN.L, CUN.R, LING.R, SOG.L, and PCUN.R. This study on DFC states explores changes in the brain functional networks of patients with MCI from the perspective of alterations in the community structures of DFC states. The findings could provide new insights into the pathological changes in the brains of MCI patients.

A unified framework of community hiding using symmetric nonnegative matrix factorization

Community detection can help us to deeply understand community structures and reveal potential social relationships among members. However, as people's concerns over the excessive mining of personal information have grown, the idea of community hiding has been proposed for privacy protection. Most existing community hiding methods based on heuristic approaches and genetic algorithms ignore the generative process of the network and cannot provide a better explanation of the rationality of the hiding mechanism. In addition, they are not simultaneously applicable to the global community, target community, or target nodes. In this study, we address the community hiding problem on three scales: global community hiding (macroscopic), target community hiding (mesoscopic), and target node hiding (microscopic). We propose a unified community hiding framework (CH-SNMF) that can be applied to all three scales. The basic idea is to use symmetric nonnegative matrix factorization to describe the network generative mechanism. It has a potential clustering capability that can mine key links and link sets during the network generation process. Therefore, they can be used to disrupt community structures with minimal perturbation budgets. The experimental results show that CH-SNMF outperforms many advanced baseline methods and effectively protects organizational and individual privacy.

Joint orthogonal symmetric non-negative matrix factorization for community detection in attribute network

Community detection is an important and challenging task in complex attribute network analysis. Symmetric non-negative matrix factorization-based methods have become promising because of their excellent ability to extract low-dimensional representations of attribute networks (which are characterized by their adjacency and attribute data). In this paper, we propose a novel community detector called joint symmetric non-negative matrix factorization model, in which the attribute homogeneity and topology similarities of an attribute network are characterized in a unified framework. An orthogonal constraint is imposed on the factor matrix to improve the accuracy of nodes’ affiliations to communities. Furthermore, a novel multi-order graph regularization was developed to preserve the intrinsic geometric structure, and an eigenvector centrality-based enhancement strategy was established to explore more comprehensive adjacency information. Extensive experiments on community detection tasks demonstrate that the proposed method performs significantly better than existing state-of-the-art methods in most benchmark complex networks.

WSNMF: Weighted Symmetric Nonnegative Matrix Factorization for attributed graph clustering

In recent times, Symmetric Nonnegative Matrix Factorization (SNMF), a derivative of Nonnegative Matrix Factorization (NMF), has surfaced as a promising technique for graph clustering. Nevertheless, when applied to attributed graph clustering, it confronts notable challenges. These include the disregard for attributed information, the oversight of geometric data point structures, and the inability to discriminate irrelevant features and data outliers. In response, we introduce an innovative extension of SNMF termed Weighted Symmetric Nonnegative Matrix Factorization (WSNMF). This method introduces node attribute similarity to compute a weight matrix, effectively bridging the gap for attributed graph clustering. Our approach incorporates graph regularization and sparsity constraints to uphold the geometric structure of data points and discern irrelevant features and data outliers. Additionally, we present an updating rule to address optimization complexities and validate algorithmic convergence. Rigorous experimentation on real-world and synthetic networks, employing well-established metrics including F-measure, RI, Modularity, Density, and entropy, substantiates the performance enhancement offered by WSNMF.

Multiview Clustering via Hypergraph Induced Semi-Supervised Symmetric Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) based multiview technique has been commonly used in multiview data clustering tasks. However, previous NMF based multiview clustering approaches fail to take advantage of a small amount of supervisory information to effectively improve the clustering performance, and are easily affected by the additional post-processing method in clustering tasks. To cope with these issues, a novel framework named multiview clustering via hypergraph induced semi-supervised symmetric NMF (MVCHSS) is proposed in this paper for multiview data clustering applications. Specifically, the proposed method has the following features: 1) a new multiview based hypergraph pairwise constraints propagation (MHPCP) algorithm is developed in MVCHSS to construct a set of informative similarity matrices, revealing the high-order relationships effectively and fully utilizing the limited pairwise constraint supervisory information among samples of each view data; 2) the obtained similarity matrices with much supervisory information are not only enforced into the symmetric NMF (SNMF) model, but also incorporated into the graph regularization for each view data; 3) the optimization problem of MVCHSS is formulated for multiview data clustering tasks to acquire a more discriminative clustering indicator matrix (or called consensus assignment matrix) without additional post-processing method. Moreover, the proof of convergence and the computational complexity for MVCHSS are presented. Extensive experiments on five multiview datasets demonstrate that the proposed MVCHSS framework outperforms several state-of-the-art multiview clustering methods.

Symmetric nonnegative matrix factorization: A systematic review

In recent years, symmetric non-negative matrix factorization (SNMF), a variant of non-negative matrix factorization (NMF), has emerged as a promising tool for data analysis. This paper mainly focuses on the theoretical idea, the basic model, the optimization method, and the variants of SNMF. The SNMF-related approaches can be generally classified into two main categories, Classic SNMFs and Extended SNMFs. The classic SNMFs contain Orthogonal SNMF, Sparse SNMF, Manifold structure based SNMF and Pairwise constraint based SNMF, besides, extended SNMFs include Self-supervised SNMF, MV-WSNMF and Multi-view SNMF. According to different classes of SNMFs, this review elaborates on the key concepts, characteristics, and current issues of these algorithms. The clustering performance of SNMF and its variants on three object image datasets is empirically compared. In addition, the effects of some algorithms for solving SNMF have been compared and the performance of similarity matrix construction methods is also compared. Finally, some open problems with SNMF are discussed.

A Bregman stochastic method for nonconvex nonsmooth problem beyond global Lipschitz gradient continuity

ABSTRACT In this paper, we consider solving a broad class of large-scale nonconvex and nonsmooth minimization problems by a Bregman proximal stochastic gradient (BPSG) algorithm. The objective function of the minimization problem is the composition of a differentiable and a nondifferentiable function, and the differentiable part does not admit a global Lipschitz continuous gradient. Under some suitable conditions, the subsequential convergence of the proposed algorithm is established. And under expectation conditions with the Kurdyka-Łojasiewicz (KL) property, we also prove that the proposed method converges globally. We also apply the BPSG algorithm to solve sparse nonnegative matrix factorization (NMF), symmetric NMF via non-symmetric relaxation, and matrix completion problems under different kernel generating distances, and numerically compare it with other algorithms. The results demonstrate the robustness and effectiveness of the proposed algorithm.

Community Detection in Multilayer Networks Via Semi-Supervised Joint Symmetric Nonnegative Matrix Factorization

Community detection plays an important role in network analysis and has attracted considerable interest from researchers. In the past few decades, various community detection algorithms have been developed for single networks. However, in the real world, relationships between nodes are often of multiple natures, such as friendship, kinship and common interests among people in social networks. These relationships can be modeled by a multilayer network. Thus, identifying communities in multilayer networks has become a challenging problem. The existing algorithms for multilayer networks only utilize the topological structure and ignore the prior information, thereby resulting in low accuracy. In this paper, by combining the graph regularization with the prior information, we propose a semi-supervised joint symmetric nonnegative matrix factorization(SSJSNMF) algorithm for community detection in multilayer networks. We use graph regularization term to penalize the latent space dissimilarity of some nodes when prior information shows that these nodes belong to same community. Then, by fusing graph regularization into a joint symmetric nonnegative matrix factorization(NMF) model, the proposed model can utilize the topological structure information and prior information simultaneously. Furthermore, we develop effective multiplicative updating rules to solve the proposed model. Finally, numerical experiments demonstrate that SSSNMF outperforms some existing algorithms.

Fast low-rank-matrix-based connection center evolution for large hyperspectral image clustering

Clustering is a common data processing technique and has been increasingly applied in remote sensing processing. In recent years, a novel clustering algorithm called connection center evolution (CCE) has been proposed. To a certain extent, CCE solves the parameter sensitivity problem, which generally exists in traditional clustering algorithms. However, CCE is not feasible for handling large hyperspectral images (HSI) due to its computational bottlenecks. That is, when the size of HSI to be processed is large, the construction of the similarity matrix and the calculation of its power have very high space complexity and time complexity, respectively. To solve this problem, we propose a novel approach for large HSI datasets, named fast low-rank-matrix-based connectivity center evolution (LMCCE). The proposed LMCCE algorithm first introduces the KNN graph into CCE to construct a sparse similarity matrix, effectively reducing its space complexity. Then, symmetric non-negative matrix factorization (SymNMF) is used for the similarity matrix, which can maintain most of the information while reducing the redundancy and noise in similarity matrix. Furthermore, by performing SymNMF on the similarity matrix, the problem of calculating the power of the similarity matrix is transformed into the problem of calculating the power of a small matrix, which significantly reduces the time complexity. Experimental results on several synthetic and real HSI datasets demonstrate that the proposed LMCCE algorithm is faster than CCE, and can handle large HSI while achieving superior performance compared to state-of-the-art clustering methods.

Hypergraph based semi-supervised symmetric nonnegative matrix factorization for image clustering

Semi-supervised symmetric nonnegative matrix factorization (SNMF) has been shown to be a significant method for both linear and nonlinear data clustering applications. Nevertheless, existing SNMF-based methods only adopt a simple graph to construct the similarity matrix, and cannot fully use the limited supervised information for the construction of the similarity matrix. To overcome the drawbacks of previous SNMF-based methods, a new semi-supervised SNMF-based method called hypergraph based semi-supervised SNMF (HSSNMF), is proposed in this paper for image clustering. Specifically, HSSNMF adopts a predefined hypergraph to build a similarity matrix for capturing the high-order relationships of samples. By exploiting a new hypergraph based pairwise constraints propagation (HPCP) algorithm, HSSNMF propagates the pairwise constraints of the limited data points to the entire data points, which can make full use of the limited supervised information and construct a more informative similarity matrix. Using the multiplicative updating algorithm, a discriminative assignment matrix can then be obtained by solving the optimization problem of HSSNMF. Moreover, analyses of the convergence, supervisory information, and computational complexity of HSSNMF are presented. Finally, extensive clustering experiments have been conducted on six real-world image datasets, and the experimental results have demonstrated the superiority of HSSNMF while compared with several state-of-the-art methods.

One step multi-view spectral clustering via joint adaptive graph learning and matrix factorization

Multi-view clustering based on graph learning has attracted extensive attention due to its simplicity and efficiency in recent years. However, there are still some issues in most of the existing graph-based multi-view clustering methods. First, most of those methods require post-processing such as K-means or spectral rotation to get the final discrete clustering result. Second, graph-based clustering methods perform clustering on a fixed input similarity graph, which could induce bad clustering results if the initial graph is with low quality. Third, these methods have high computation cost, which hinders them for dealing with large-scale data. In order to solve these problems, we propose a multi-view spectral clustering method via joint Adaptive Graph Learning and Matrix Factorization (AGLMF). In this method, to reduce computational cost, we adopt the anchor-based strategy to construct the input similarity graphs. Then, we use the l1-norm to learn a high quality similarity graph adaptively from original similarity graphs which can make the final graph more robust than original ones. In addition, AGLMF uses symmetric non-negative matrix factorization to learn the final clustering indicators which can show the final consistent clustering result directly. Finally, experimental results on multiple multi-view datasets validate the effectiveness of the proposed algorithm when compared with previous multi-view spectral clustering algorithms. The demo code of this work is publicly available at https://github.com/theywq/AGLMF.git.

An improved multi-view spectral clustering based on tissue-like P systems

Multi-view spectral clustering is one of the multi-view clustering methods widely studied by numerous scholars. The first step of multi-view spectral clustering is to construct the similarity matrix of each view. Consequently, the clustering performance will be greatly affected by the quality of the similarity matrix of each view. To solve this problem well, an improved multi-view spectral clustering based on tissue-like P systems is proposed in this paper. The optimal per-view similarity matrix is generated in an iterative manner. In addition, spectral clustering is combined with the symmetric nonnegative matrix factorization method to directly output the clustering results to avoid the secondary operation, such as k-means or spectral rotation. Furthermore, improved multi-view spectral clustering is integrated with the tissue-like P system to enhance the computational efficiency of the multi-view clustering algorithm. Extensive experiments verify the effectiveness of this algorithm over other state-of-the-art algorithms.

Symmetric Nonnegative Matrix Factorization for Vertex Centrality in Complex Networks

Symmetric Nonnegative Matrix Factorization for Vertex Centrality in Complex Networks

Prediction of brain age based on the community structure of functional networks

Studies have demonstrated that brain functional networks (BFNs) and machine learning are promising methods for predicting brain age. However, many studies ignore the important structural properties of BFNs and the feature alterations of community structure in healthy aging brains. Given this issue, we developed a novel scheme to predict brain age based on resting-state functional magnetic resonance imaging (rs-fMRI). Firstly, motivated by the fact that the brain is organized with sparsity, modular and other properties, we established a novel BFNs model using the nonnegative block diagonal representation (NBDR) with block diagonal matrix-induced regularization. Secondly, the collective sparse symmetric nonnegative matrix factorization (cssNMF) was adopted to detect the individual-level community strengths as a mesoscale network topology structure, which combined with small-scale and large-scale network topologies, were used as features. Finally, brain age was predicted by using six different models with a machine learning framework based on the extracted features. Experimental results on both simulated and real data revealed that the novel BFNs method was superior to other compared methods in connectivity, and the feature extraction scheme proposed performed well in predicting brain age. This research provides a novel insight into the construction of BFNs and the feature extraction of brain signals, which is useful to understand the mechanism of normal brain aging.

Multi-Constrained Embedding for Accurate Community Detection on Undirected Networks

A Symmetric Non-negative Matrix Factorization (SNMF)-based network embedding model adopts a unique Latent Factor (LF) matrix for describing the symmetry of an undirected network, which reduces its representation ability to the target network and thus resulting in accuracy loss when performing community detection. To address this issue, this paper proposes a new undirected network embedding model, i.e., Alternating Direction Method of Multipliers ( <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u> DMM)-based, <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</u> odularity, <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</u> ymmetry and <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</u> onnegativity-constrained <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</u> mbedding (AMSNE), which can be applicable to undirected, weighted or unweighted networks. It relies on two-fold ideas: a) Introducing the symmetry constraints into the model to correctly describe the symmetric of an undirected network without accuracy loss; and b) Adopting the ADMM principle to efficiently solve its constrained objective. Extensive experiments on eight real-world networks strongly evidence that the proposed AMSNE outperform several state-of-the-art models, making it suitable for real applications.

Symmetry and Nonnegativity-Constrained Matrix Factorization for Community Detection

Dear Editor, This letter presents a novel symmetry and nonnegativity-constrained matrix factorization (SNCMF)-based community detection model on undirected networks such as a social network. Community is a fundamental characteristic of a network, making community detection a vital yet thorny issue in network representation. Owing to its high interpretability and scalability, a symmetric nonnegative matrix factorization (SNMF) model is frequently adopted to address this issue. However, it adopts a unique latent factor (LF) matrix for representing an undirected network's symmetry, which leads to a reduced latent space that impairs its representation learning ability. Motivated by this discovery, the proposed SNCMF model innovatively adopts the following three-fold ideas: 1) Leveraging multiple LF matrices to represent a network, thereby enhancing its representation learning ability; 2) Introducing a symmetry regularization term that implies the equality constraint between multiple LF matrices to illustrate the network's symmetry; and 3) Incorporating graph regularization into the model to preserve the network's intrinsic geometry. Experimental results on several real-world networks indicate that the proposed SNCMF-based community detector outperforms the benchmark and state-of-the-art models in achieving highly-accurate community detection results.

Constraint-Induced Symmetric Nonnegative Matrix Factorization for Accurate Community Detection

As a fundamental characteristic of an undirected network, community reveals its networking organization and functional mechanisms, making community detection be a highly-interesting issue in network representation learning. With great interpretability, a symmetric and nonnegative matrix factorization (SNMF)-based approach is frequently adopted to tackle this issue. However, it only adopts a unique feature matrix for describing the symmetry of an undirected network, which unfortunately results in a reduced feature space that evidently impairs its representation learning ability. Motivated by this discovery, this paper proposes a novel Constraint-induced Symmetric Nonnegative Matrix Factorization (C-SNMF) model that adopts three-fold ideas: a) Representing a target undirected network with multiple latent feature matrices, thus preserving its representation learning capacity; b) Incorporating a symmetry-regularizer into its objective function, which preserves the symmetry of the learnt low-rank approximation to the adjacency matrix, thereby making the resultant detector precisely illustrate the target network's symmetry; and c) Introducing a graph-regularizer that preserves local invariance of the network's intrinsic geometry into its learning objective, thus making the achieved detector well-aware of community structure within the target network. Note that the regularization coefficients are selected according to the modularity of the learnt community structure on the training data only, thereby greatly improving the achieved model's practical significance for real applications. Experimental results on six real-world networks demonstrate that the proposed C-SNMF model significantly outperforms the benchmarks and state-of-the-art models in achieving highly-accurate community detection results.