Graph Regularization Term Research Articles

Graph Regularized Sparse L2,1 Semi‐Nonnegative Matrix Factorization for Data Reduction

ABSTRACTNon‐negative Matrix Factorization (NMF) is an effective algorithm for multivariate data analysis, including applications to feature selection, pattern recognition, and computer vision. Its variant, Semi‐Nonnegative Matrix Factorization (SNF), extends the ability of NMF to render parts‐based data representations to include mixed‐sign data. Graph Regularized SNF builds upon this paradigm by adding a graph regularization term to preserve the local geometrical structure of the data space. Despite their successes, SNF‐related algorithms to date still suffer from instability caused by the Frobenius norm due to the effects of outliers and noise. In this paper, we present a new SNF algorithm that utilizes the noise‐insensitive norm. We provide monotonic convergence analysis of the SNF algorithm. In addition, we conduct numerical experiments on three benchmark mixed‐sign datasets as well as several randomized mixed‐sign matrices to demonstrate the performance superiority of SNF over conventional SNF algorithms under the influence of Gaussian noise at different levels.

Identification of dynamic networks community by fusing deep learning and evolutionary clustering

Community detection is a critical component of network analysis and a hot topic in social computing. Detecting community structure in dynamic networks has important theoretical and practical implications for understanding the intrinsic function of networks and predicting network behavior. However, the majority of existing dynamic community detection methods adopt shallow models, which have limited ability to excavate complex non-linear structures and tend to generate undesirable community structures. In order to obtain an accurate and robust community structure in dynamic networks, we are inspired by network representation learning and utilize the deep learning to detect evolving communities in dynamic networks. In this paper, we propose a novel dynamic community detection method by fusing Deep Learning and Evolutionary Clustering (DLEC). This work attempts to combine deep learning and evolutionary clustering into a unified framework. First, we propose a matrix construction strategy to fully reveal the inherent community structures via the underlying community memberships. Then, we develop a novel multi-layer deep autoencoder framework that consists of multiple non-linear functions to extract the latent deep representation of the dynamic network. Based on the evolutionary clustering framework, a graph regularization term is introduced to ensure the smoothness of the community evolution. Finally, we employ the K-means clustering algorithm on the low-dimensional network space to obtain the community structure. Extensive experimental results on synthetic and real-world networks show that the proposed DLEC algorithm can effectively detect high-quality communities in dynamic networks.

Selective multiple kernel fuzzy clustering with locality preserved ensemble

Multiple kernel fuzzy clustering (MKFC) has demonstrated promising performance in capturing the non-linear relationships within data. However, its effectiveness relies heavily on the appropriate selection of the fuzzification coefficient and kernel functions. To address this challenge, this paper proposes a novel clustering ensemble approach for improving the robustness and accuracy of the MKFC algorithm. The proposed method employs a multi-objective evolutionary optimization approach to heuristically select the optimal fuzzification and kernel coefficients. By employing the selected coefficients, the MKFC algorithm generates a diverse set of accurate candidate clustering results. Subsequently, a locality preserved ensemble mechanism is introduced to derive the final partition matrix, ensuring that all candidate clusterings within the Pareto non-dominated set contribute to the final consensus matrix. Moreover, this mechanism incorporates the knowledge about the locality of the dataset via a graph regularization term, thereby further enhancing the clustering performance. Comprehensive experiments conducted on widely adopted benchmark datasets demonstrate the superiority of the proposed method over the state-of-the-art approaches.

Diverse joint nonnegative matrix tri-factorization for attributed graph clustering

Cluster analysis of attributed graphs is a demanding and challenging task in the analysis of network-structured data. It involves learning node representation by leveraging both node attributes and the topological structure of the graph, aiming to accomplish effective clustering. Typically, existing methods fuse the topological and non-topological information by learning a consensus representation, often resulting in redundancy and overlooking their inherent distinctions. To address this issue, this paper proposes the Diverse Joint Nonnegative Matrix Tri-Factorization (Div-JNMTF), an embedding based model to detect communities in attributed graphs. The novel JNMTF model attempts to extract two distinct node representations from topological and non-topological data. Simultaneously, a diversity regularization technique utilizing the Hilbert–Schmidt Independence Criterion (HSIC) is employed. Its objective is to reduce redundant information in the node representations while encouraging the distinct contributions of both types of information. In addition, two graph regularization terms are introduced to preserve the local structures in the topological and attribute representation spaces. The Div-JNMTF model is optimized by developing an iterative optimization approach. By conducting thorough experiments on four synthetic and eight real-world attributed graph datasets, it has been demonstrated that the proposed model excels in accurately detecting attributed communities and surpasses the performance of existing methods.

Consensus Subspace Graph Regularization based on prior information for multiplex network clustering

Multiplex network clustering can identify the common cluster structure shared by all layers, which is of great significance for downstream research such as personalized recommendations of social relationships and the mining of social media communication behaviors. Traditional multiplex network clustering methods rely only on the topological structure and are suitable for networks with clear structures. However, real-world networks often suffer from sparse connectivity and noisy edges. In addition, the current methods based on prior information only use the obtained consensus prior information as a preprocessing method for multiplex networks, and do not fully utilize and integrate the prior information in the clustering process, resulting in a low accuracy. To solve the problem of insufficient utilization of prior information, we propose a Consensus Subspace Graph Regularization (CSGR) approach for multiplex network clustering, which integrates topological information with the consensus prior information of a network. We first construct the consensus prior matrix of a network by a non-overlapping greedy search method, which represents a subset of the network that consists of nodes with high edge density. Then we construct a graph regularization term, which encodes the consensus prior information, to optimize the generation of a joint low-dimensional representation within the consensus subspace for the entire multiplex network. Finally, we employ the consensus prior matrix to denoise each network layer, thereby yielding a precise low-dimensional representation of each layer for consensus subspace fusion. In addition, comprehensive experiments have demonstrated that CSGR improves the clustering accuracy of real-world networks by an average of 1.56%.

Discriminative Subspace Learning With Adaptive Graph Regularization

Abstract Many subspace learning methods based on low-rank representation employ the nearest neighborhood graph to preserve the local structure. However, in these methods, the nearest neighborhood graph is a binary matrix, which fails to precisely capture the similarity between distinct samples. Additionally, these methods need to manually select an appropriate number of neighbors, and they cannot adaptively update the similarity graph during projection learning. To tackle these issues, we introduce Discriminative Subspace Learning with Adaptive Graph Regularization (DSL_AGR), an innovative unsupervised subspace learning method that integrates low-rank representation, adaptive graph learning and nonnegative representation into a framework. DSL_AGR introduces a low-rank constraint to capture the global structure of the data and extract more discriminative information. Furthermore, a novel graph regularization term in DSL_AGR is guided by nonnegative representations to enhance the capability of capturing the local structure. Since closed-form solutions for the proposed method are not easily obtained, we devise an iterative optimization algorithm for its resolution. We also analyze the computational complexity and convergence of DSL_AGR. Extensive experiments on real-world datasets demonstrate that the proposed method achieves competitive performance compared with other state-of-the-art methods.

Unifying Graph Neural Networks with a Generalized Optimization Framework

Graph Neural Networks (GNNs) have received considerable attention on graph-structured data learning for a wide variety of tasks. The well-designed propagation mechanism, which has been demonstrated effective, is the most fundamental part of GNNs. Although most of the GNNs basically follow a message passing manner, little effort has been made to discover and analyze their essential relations. In this paper, we establish a surprising connection between different propagation mechanisms with an optimization problem. We show that despite the proliferation of various GNNs, in fact, their proposed propagation mechanisms are the optimal solutions of a generalized optimization framework with a flexible feature fitting function and a generalized graph regularization term. Actually, the optimization framework can not only help understand the propagation mechanisms of GNNs, but also open up opportunities for flexibly designing new GNNs. Through analyzing the general solutions of the optimization framework, we provide a more convenient way for deriving corresponding propagation results of GNNs. We further discover that existing works usually utilize naïve graph convolutional kernels for feature fitting function, or just utilize one-hop structural information (original topology graph) for graph regularization term. Correspondingly, we develop two novel objective functions considering adjustable graph kernels showing low-pass or high-pass filtering capabilities and one novel objective function considering high-order structural information during propagation respectively. Extensive experiments on benchmark datasets clearly show that the newly proposed GNNs not only outperform the state-of-the-art methods but also have good ability to alleviate over-smoothing, and further verify the feasibility for designing GNNs with the generalized unified optimization framework.

Robust sparse concept factorization with graph regularization for subspace learning

Concept factorization (CF) is a powerful tool in subspace learning. Recently graph-based CF and local coordinate CF have been proposed to exploit the intrinsic geometrical structure of data, and have been shown quite successful in improving performance. However, these methods have limited robustness and might be sensitive to noises and disturbances in practical applications. In this paper, we propose a novel robust sparse CF framework (RSCF) for subspace learning. Specifically, we present a robust loss function to effectively eliminate the impact of the large outliers. The local coordinate constraint and the graph regularization term are incorporated into RSCF to simultaneously guarantee the sparsity of the coefficient matrix and maintain the local structure of the data. We prove that the local coordinate constraint implies the orthogonality of the coefficient matrix. By using the half-quadratic optimization technique, we transform the objective function of RSCF into a quadratic form and provide the iterative updating rules and the convergence analysis. Extensive experiments demonstrate the robustness and superiority of the proposed RSCF in comparison to state-of-the-art CF methods.

Intelligent fault diagnosis of ultrasonic motors based on graph-regularized CNN-BiLSTM

The ultrasonic motor (USM) is peculiarly prone to failure due to continuous high-frequency friction-related power transfer, whose failure mechanisms are remarkably different from traditional induction motors. Intelligent fault diagnosis provides a way to alarm and avoid catastrophic losses proactively. However, previous studies using deep learning usually ignore the inherent geometric structure of the signal distribution. This paper proposes an intelligent multi-signal fault diagnosis framework for USMs to restore the linear or nonlinear manifold structure by preserving the internal structure by integrating graph regularization with deep neural networks. Firstly, the one-dimensional CNN to learn spatial correlations and BiLSTM to exploit temporal dependencies are coalesced to build the deep neural network. Then, an improved k-nearest neighbor graph is proposed to protect the geometric structure information and force the latent features to be more concentrated within their classes. Moreover, the layer in the deep architecture to integrate graph regularization is designed to reduce computation cost, and an adaptive decay strategy is considered to adjust the coefficient of graph regularized automatically. A two-stage training algorithm is developed by considering the time to calculate the graph regularization term. Finally, the proposed multi-signal fault diagnosis framework is validated using datasets from the fault injection experiment of similar USMs in China’s Yutu rover of Chang’e lunar probe. Experimental results show that the proposed method can effectively discriminate different fault types.

Projected cross-view learning for unbalanced incomplete multi-view clustering

Incomplete multi-view clustering (IMVC) aims to partition samples into different groups for datasets with missing samples. The primary goal of IMVC is to effectively address the challenge posed by missing information in clustering analysis. Most existing IMVC methods focus on balanced incomplete multi-view data, assuming a uniform missing rate across all views. However, this assumption does not accurately reflect real-life scenarios. In reality, unbalanced incomplete multi-view data, characterized by varying missing rates among different views, is more prevalent. This presents significant challenges to the clustering process, as varying missing rates can lead to information imbalance. To address these challenges, this paper introduces a novel approach called projected cross-view learning for unbalanced incomplete multi-view clustering (PCL_UIMVC). Specifically, a reconstruction term is integrated, which leverages the information from the existing samples to facilitate the completion of the unbalanced incomplete multi-view data. Next, a projection matrix is incorporated into the model to harmonize feature dimensions across views, mitigating the impact of information imbalance. Then, a graph regularization term is integrated to preserve the geometric structure of the original data. Finally, an iterative algorithm is developed to solve the proposed model. Extensive experiments on eight standard datasets, featuring various rates of missing data, validate the superior clustering performance of the proposed method.

SAGN: Sparse Adaptive Gated Graph Neural Network With Graph Regularization for Identifying Dual-View Brain Networks.

Due to the absence of a gold standard for threshold selection, brain networks constructed with inappropriate thresholds risk topological degradation or contain noise connections. Therefore, graph neural networks (GNNs) exhibit weak robustness and overfitting problems when identifying brain networks. Furthermore, existing studies have predominantly focused on strongly coupled connections, neglecting substantial evidence from other intricate systems that highlight the value of weakly coupled connections. Consequently, the potential of weakly coupled brain networks remains untapped. In this study, we pioneeringly construct weakly coupled brain networks and validate their values in emotion identification tasks. Subsequently, we propose a sparse adaptive gated GNN (SAGN) that can simultaneously perceive the valuable topology of dual-view (i.e., strongly coupled and weakly coupled) brain networks. The SAGN contains a sparse adaptive global receptive field. Moreover, SAGN employs a gated mechanism with feature enhancement and adaptive noise suppression capabilities. To address the lack of inductive bias and the large capacity of SAGN, a graph regularization term built with prior topology of dual-view brain networks is introduced to enhance generalization. Besides a public dataset (SEED), we also built a custom dataset (MuSer) with 60 subjects to evaluate weakly coupled brain networks' value and validate the SAGN's performance. Experiments demonstrate that brain physiological patterns associated with different emotional states are separable and rooted in weakly coupled brain networks. In addition, SAGN exhibits excellent generalization and robustness in identifying brain networks.

Robust Nonnegative Matrix Factorization With Self-Initiated Multigraph Contrastive Fusion.

Graph regularized nonnegative matrix factorization (GNMF) has been widely used in data representation due to its excellent dimensionality reduction. When it comes to clustering polluted data, GNMF inevitably learns inaccurate representations, leading to models that are unusually sensitive to outliers in the data. For example, in a face dataset, obscured by items such as a mask or glasses, there is a high probability that the graph regularization term incorrectly describes the association relationship for that sample, resulting in an incorrect elicitation in the matrix factorization process. In this article, a novel self-initiated unsupervised subspace learning method named robust nonnegative matrix factorization with self-initiated multigraph contrastive fusion (RNMF-SMGF) is proposed. RNMF-SMGF is capable of creating samples with different angles and learning different graph structures based on these different angles in a self-initiated method without changing the original data. In the process of subspace learning guided by graph regularization, these different graph structures are fused into a more accurate graph structure, along with entropy regularization, L2,1/2 -norm constraints to facilitate the robust learning of the proposed model and the formation of different clusters in the low-dimensional space. To demonstrate the effectiveness of the proposed model in robust clustering, we have conducted extensive experiments on several benchmark datasets and demonstrated the effectiveness of the proposed method. The source code is available at: https://github.com/LstinWh/RNMF-SMGF/.

Non-negative Tucker decomposition with graph regularization and smooth constraint for clustering

Non-negative Tucker decomposition (NTD) and its graph regularized extensions are the most popular techniques for representing high-dimensional non-negative data, which are typically found in a low-dimensional sub-manifold of ambient space, from a geometric perspective. Therefore, the performance of the graph-based NTD methods relies heavily on the low-dimensional representation of the original data. However, most existing approaches treat the last factor matrix in NTD as a low-dimensional representation of the original data. This treatment leads to the loss of the original data’s multi-linear structure in the low-dimensional subspace. To remedy this defect, we propose a novel graph regularized Lp smooth NTD (GSNTD) method for high-dimensional data representation by incorporating graph regularization and an Lp smoothing constraint into NTD. The new graph regularization term constructed by the product of the core tensor and the last factor matrix in NTD, and it is used to uncover hidden semantics while maintaining the intrinsic multi-linear geometric structure of the data. The addition of the Lp smoothing constraint to NTD may produce a more accurate and smoother solution to the optimization problem. The update rules and the convergence of the GSNTD method are proposed. In addition, a randomized variant of the GSNTD algorithm based on fiber sampling is proposed. Finally, the experimental results on four standard image databases show that the proposed method and its randomized variant have better performance than some other state-of-the-art graph-based regularization methods for image clustering.

Side-constrained graph fusion for semi-supervised multi-view clustering

In these years, semi-supervised learning arouses ongoing attentions due to its appealing avail and economic expenditure of guiding model training. Semi-supervised multi-view clustering takes advantages of heterogeneous features and a small number of side constraints (i.e., must-link constraints and cannot-link constraints) to partition data points. However, most of existing approaches are very limited in absorbing available side constraints since it is difficult to optimize cannot-link constraints in a provable way, and thus greatly waste the value of prior information. To address it, we innovatively pose a side-constrained multi-view graph clustering method, where the pairwise constraints are flexibly incorporated into the multiple graph fusion framework. The technique definitely formulates the pairwise constraints in the graph clustering model by designing a semi-supervised graph regularization term. In this way, the structured optimal graph that satisfies multiple perspectives and specified pairwise relations is obtained. By virtue of graph fusion, the self-adaptive weight of each single-view is optimally determined without partiality. We demonstrate theoretical feasibility of the proposed method. Extensive experimental results in four multi-view data sets witness our superiority compared to the state-of-the-art approaches.

Elastic deep autoencoder for text embedding clustering by an improved graph regularization

Text clustering is a task for grouping extracted information of the text in different clusters, which has many applications in recommender systems, sentiment analysis, and more. Deep learning-based methods have become increasingly popular due to their high accuracy in identifying nonlinear structures. They usually consist of two major parts: dimensionality reduction and clustering. Autoencoders are simple unsupervised neural networks used for better representation of low-dimensional data and have shown good performance in dealing with non-linear features. However, while they utilize the Frobenius norm to deal well with Gaussian noise, they are sensitive to outlier data and Laplacian noise. In this paper, a deep autoencoder with an adapted elastic loss for text embedding clustering (EDA-TEC) is proposed. The elastic loss is a combination of the Frobenius norm and L2,1-norm to consider both types of noises. Additionally, to maintain the high-dimensional data geometric structure, a modified graph regularization term based on the weighted cosine similarity measure is used. EDA-TEC also improves clustering results by considering the sparsity regularization of the manifold representation data. In this jointly end-to-end deep learning model, better representation and text clustering results are achieved with high accuracy on common datasets compared to existing methods.11https://github.com/safinal/text-embedding-clustering.

Auto-adjustable hypergraph regularized non-negative matrix factorization for image clustering

Non-negative matrix factorization (NMF) is an effective method for image clustering. However, relatively fixed graph regularization terms and loss functions have been adopted by recently proposed variants of NMF, and their clustering performance can be improved by incorporating configurable parameters. In this paper, an auto-adjustable hypergraph regularized non-negative matrix factorization (AHRNMF) algorithm was proposed. In the AHRNMF framework, we proposed a piecewise loss function and an innovative auto-adjustable hypergraph. The loss function incorporates two adaptive parameters, harmonizing reconstruction error and anti-outlier efficacy. Hypergraph construction relies on the calculation of two k-nearest neighbors (KNN) with different scales. Furthermore, an KNN-based algorithm was developed to assist AHRNMF in achieving auto-adjustment, which can automatically detect outliers without determining the number of clusters in advance. It was demonstrated by extensive experiments that the proposed AHRNMF outperforms other state-of-the-art methods.

Multi-view clustering via label-embedded regularized NMF with dual-graph constraints

Nonnegative matrix factorization (NMF) methods have achieved remarkable performances in multi-view clustering due to their effectiveness and efficiency. To better obtain a low-dimensional common representation, the limited labels and the geometric structure of the multi-view data should be fully utilized in clustering. In this work, we introduce a novel multi-view learning approach, dubbed label-embedded regularized NMF with dual-graph constraints (LeNMF-DC), for clustering. Our proposed LeNMF-DC approach mainly utilizes matrix factorization to obtain a low-dimensional common representation of the multi-view data, in which the prior knowledge hidden in data can be fully explored. Specifically, we construct three graph regularization terms to preserve the manifold structure in the data, feature and label space, respectively. Moreover, we take advantage of the labels of the labeled samples without additional parameters. In addition, we develop an alternate iterative optimization scheme to solve the model of LeNMF-DC and then show its convergence rate. Compared with traditional multi-view clustering approaches, the labels of unlabeled samples in our proposed LeNMF-DC approach are assigned by the label constraint matrix rather than the clustering algorithm, and thus it avoids performance loss during the clustering. Experimental results on four benchmark datasets manifest that our LeNMF-DC approach can achieve superior performances than several state-of-the-art approaches in multi-view clustering.

Community Detection in Multilayer Networks Via Semi-Supervised Joint Symmetric Nonnegative Matrix Factorization

Community detection plays an important role in network analysis and has attracted considerable interest from researchers. In the past few decades, various community detection algorithms have been developed for single networks. However, in the real world, relationships between nodes are often of multiple natures, such as friendship, kinship and common interests among people in social networks. These relationships can be modeled by a multilayer network. Thus, identifying communities in multilayer networks has become a challenging problem. The existing algorithms for multilayer networks only utilize the topological structure and ignore the prior information, thereby resulting in low accuracy. In this paper, by combining the graph regularization with the prior information, we propose a semi-supervised joint symmetric nonnegative matrix factorization(SSJSNMF) algorithm for community detection in multilayer networks. We use graph regularization term to penalize the latent space dissimilarity of some nodes when prior information shows that these nodes belong to same community. Then, by fusing graph regularization into a joint symmetric nonnegative matrix factorization(NMF) model, the proposed model can utilize the topological structure information and prior information simultaneously. Furthermore, we develop effective multiplicative updating rules to solve the proposed model. Finally, numerical experiments demonstrate that SSSNMF outperforms some existing algorithms.

Learnable Graph-Regularization for Matrix Decomposition

Low-rank approximation models of data matrices have become important machine learning and data mining tools in many fields, including computer vision, text mining, bioinformatics, and many others. They allow for embedding high-dimensional data into low-dimensional spaces, which mitigates the effects of noise and uncovers latent relations. In order to make the learned representations inherit the structures in the original data, graph-regularization terms are often added to the loss function. However, the prior graph construction often fails to reflect the true network connectivity and the intrinsic relationships. In addition, many graph-regularized methods fail to take the dual spaces into account. Probabilistic models are often used to model the distribution of the representations, but most of previous methods often assume that the hidden variables are independent and identically distributed for simplicity. To this end, we propose a learnable graph-regularization model for matrix decomposition (LGMD), which builds a bridge between graph-regularized methods and probabilistic matrix decomposition models for the first time. LGMD incorporates two graphical structures (i.e., two precision matrices) learned in an iterative manner via sparse precision matrix estimation and is more robust to noise and missing entries. Extensive numerical results and comparison with competing methods demonstrate its effectiveness.

Adaptive graph regularization and self-expression for noise-aware feature selection

Many traditional unsupervised feature selection algorithms utilize manifold information to mine the local structure of the data. However, the noise existing in the raw data reduces the accuracy of the manifold information of data, which affects the learning effect of the entire algorithm. In order to solve the above problems and more fully find the internal structure inside the data, this paper proposes an adaptive graph regularization and self-expression for noise-aware feature selection (ASNFS). Firstly, the algorithm adopts non-negative matrix factorization to decompose the raw data matrix, and utilizes the low-dimensional matrix generated after decomposition to replace the raw high-dimensional data matrix. This allows the algorithm to reveal some internal structural information of the raw data while reducing the dimensionality of the data. ASNFS also introduces the orthogonal basis clustering with excellent clustering effect, and the interpretability of the algorithm is enhanced. Secondly, in addition to preserving the manifold information in the low-dimensional projection subspace, the algorithm also preserves the manifold information in the non-negative matrix factorization subspace. Meanwhile, the adaptive graph regularization term added to the objective function enables the algorithm to continuously update the similarity matrix. It can effectively remove the noise inside the raw data, and prevent the occurrence of over-fitting phenomenon of the experimental results caused by the fixed similarity matrix. Finally, the similarity matrix retains the local structure information of the data with each iteration, and the results of the feature selection are reused for the construction of the similarity matrix. ASNFS adopts an alternate iterative method to optimize the objective function, which is simple and effective. Then, the algorithm complexity and convergence are analyzed. ASNFS is compared with seven feature selection algorithms on nine datasets, and the experimental results reflect the effectiveness of ASNFS in feature selection.