Novel Nonnegative Matrix Factorization Research Articles

A novel nonnegative matrix factorization-based model for attributed graph clustering by incorporating complementary information

Attributed graph clustering is a prominent research area, catering to the increasing need for understanding real-world systems by uncovering exhaustive meaningful latent knowledge from heterogeneous spaces. Therefore, the critical challenge of this problem is the strategy used to extract and integrate meaningful heterogeneous information from structure and attribute sources. To this end, in this paper, we propose a novel Nonnegative Matrix Factorization (NMF)-based model for attributed graph clustering. In this method, firstly, we filter structure and attribute spaces from noise and irrelevant information for clustering by applying Symmetric NMF and NMF during the clustering task, respectively. Then, to overcome the heterogeneity of discovered partitions from spaces, we suggest a new regularization term to inject the complementary information from the attribute partition into the structure by transforming them into their pairwise similarity spaces, which are homogeneous. Simultaneously, by setting orthogonality constraints on the discovered communities, we encourage the representation of distinct and non-overlapping communities within the attributed graph. Finally, we collect all these terms in a unified framework to learn a meaningful partition containing consensus and complementary information from structure and attributes. Then a new iterative multiplicative updating strategy is proposed to solve the proposed model, and its convergence is proven theoretically. Our experiments on the nine popular real-world networks illustrate the supremacy of our methods among eleven widely recognized and stat-of-the-arts attributed graph clustering methods in terms of accurately matching the ground truth and quality-based metrics.

CSMVC: A Multiview Method for Multivariate Time-Series Clustering.

Multivariate time-series (MTS) clustering is a fundamental technique in data mining with a wide range of real-world applications. To date, though some approaches have been developed, they suffer from various drawbacks, such as high computational cost or loss of information. Most existing approaches are single-view methods without considering the benefits of mutual-support multiple views. Moreover, due to its data structure, MTS data cannot be handled well by most multiview clustering methods. Toward this end, we propose a consistent and specific non-negative matrix factorization-based multiview clustering (CSMVC) method for MTS clustering. The proposed method constructs a multilayer graph to represent the original MTS data and generates multiple views with a subspace technique. The obtained multiview data are processed through a novel non-negative matrix factorization (NMF) method, which can explore the view-consistent and view-specific information simultaneously. Furthermore, an alternating optimization scheme is proposed to solve the corresponding optimization problem. We conduct extensive experiments on 13 benchmark datasets and the results demonstrate the superiority of our proposed method against other state-of-the-art algorithms under a wide range of evaluation metrics.

CDCN: A New NMF-Based Community Detection Method with Community Structures and Node Attributes

Community discovery can discover the community structure in a network, and it provides consumers with personalized services and information pushing. It plays an important role in promoting the intelligence of the network society. Most community networks have a community structure whose vertices are gathered into groups which is significant for network data mining and identification. Existing community detection methods explore the original network topology, but they do not make the full use of the inherent semantic information on nodes, e.g., node attributes. To solve the problem, we explore networks by considering both the original network topology and inherent community structures. In this paper, we propose a novel nonnegative matrix factorization (NMF) model that is divided into two parts, the community structure matrix and the node attribute matrix, and we present a matrix updating method to deal with the nonnegative matrix factorization optimization problem. NMF can achieve large-scale multidimensional data reduction processing to discover the internal relationships between networks and find the degree of network association. The community structure matrix that we proposed provides more information about the network structure by considering the relationships between nodes that connect directly or share similar neighboring nodes. The use of node attributes provides a semantic interpretation for the community structure. We conduct experiments on attributed graph datasets with overlapping and nonoverlapping communities. The results of the experiments show that the performances of the F1-Score and Jaccard-Similarity in the overlapping community and the performances of normalized mutual information (NMI) and accuracy (AC) in the nonoverlapping community are significantly improved. Our proposed model achieves significant improvements in terms of its accuracy and relevance compared with the state-of-the-art approaches.

Multikernel Clustering via Non-Negative Matrix Factorization Tailored Graph Tensor Over Distributed Networks

Next-generation wireless networks are witnessing an increasing number of clustering applications, and produce a large amount of non-linear and unlabeled data. In some degree, single kernel methods face the challenging problem of kernel choice. To overcome this problem for non-linear data clustering, multiple kernel graph-based clustering (MKGC) has attracted intense attention in recent years. However, existing MKGC methods suffer from two common problems: (1) they mainly aim to learn a consensus kernel from multiple candidate kernels, slight affinity graph learning, such that cannot fully exploit the underlying graph structure of non-linear data; (2) they disregard the high-order correlations between all base kernels, which cannot fully capture the consistent and complementary information of all kernels. In this paper, we propose a novel non-negative matrix factorization (NMF) tailored graph tensor MKGC method for non-linear data clustering, namely TMKGC. Specifically, TMKGC integrates NMF and graph learning together in kernel space so as to learn multiple candidate affinity graphs. Afterwards, the high-order structure information of all candidate graphs is captured in a 3-order tensor kernel space by introducing tensor singular value decomposition based tensor nuclear norm, such that an optimal affinity graph can be obtained subsequently. Based on the alternating direction method of multipliers, the effective local and distributed solvers are elaborated to solve the proposed objective function. Extensive experiments have demonstrated the superiority of TMKGC compared to the state-of-the-art MKGC methods.

FLGAI: a unified network embedding framework integrating multi-scale network structures and node attribute information

Network embedding is an effective method aiming to learn the low-dimensional vector representation of nodes in networks, which has been widely used in various network analytic tasks such as node classification, node clustering, and link prediction. The objective of network embedding is to capture the structural information and inherent characteristics of the network as much as possible in the low-dimensional vector representation. However, the majority of the existing network embedding methods merely exploited the microscopic proximity of the network structure to learn the node representation, which tend to generate sub-optimal network representation. In this paper, we propose a novel nonnegative matrix factorization (NMF) based network representation learning framework called FLGAI, which jointly integrates the local network structure, global network structure, and attribute information to learn the network representation. First, we employ the first-order proximity and second-order proximity jointly to preserve the local network structure. Then, the community structure is introduced to preserve the global network structure. Third, we exploit the node attribute information to capture the node characteristics. To preserve the structural information and the network node attributes simultaneously, we formulate their consensus relationships and optimize them jointly in a unified NMF framework to derive the final network representation. To evaluate the effectiveness of our model, we conduct extensive experiments on six real-world datasets and the empirical results demonstrate the superior performance of the proposed method over the state-of-the-art approaches in both node classification and node clustering tasks.

A novel non-negative matrix factorization technique for decomposition of Chinese characters with application to secret sharing

The decomposition of Chinese characters is difficult and has been rarely investigated in the literature. In this paper, we propose a novel non-negative matrix factorization (NMF) technique to decompose a Chinese character into several graphical components without considering the strokes of the character or any semantic or phonetic properties of the components. Chinese characters can usually be represented as binary images. However, traditional NMF is only suitable for representing general gray-level or color images. To decompose a binary image using NMF, we force all of the elements of the two matrices (obtained by factorizing the binary image/matrix to be decomposed) as close to 0 or 1 as possible. As a result, a Chinese character can be efficiently decomposed into several components, where each component is semantically unreadable. Moreover, our NMF-based Chinese character decomposition method is suitable for applications in visual secret sharing by distributing the shares (different character components) among multiple parties, so that only when the parties are taken together with their respective shares can the secret (the original Chinese character(s)) be reconstructed. Experimental results have verified the decomposition performance and the feasibility of the proposed method.

Non-negative matrix factorization based modeling and training algorithm for multi-label learning

Multi-label learning is more complicated than single-label learning since the semantics of the instances are usually overlapped and not identical. The effectiveness of many algorithms often fails when the correlations in the feature and label space are not fully exploited. To this end, we propose a novel non-negative matrix factorization (NMF) based modeling and training algorithm that learns from both the adjacencies of the instances and the labels of the training set. In the modeling process, a set of generators are constructed, and the associations among generators, instances, and labels are set up, with which the label prediction is conducted. In the training process, the parameters involved in the process of modeling are determined. Specifically, an NMF based algorithm is proposed to determine the associations between generators and instances, and a non-negative least square optimization algorithm is applied to determine the associations between generators and labels. The proposed algorithm fully takes the advantage of smoothness assumption, so that the labels are properly propagated. The experiments were carried out on six set of benchmarks. The results demonstrate the effectiveness of the proposed algorithms.

Multi-view non-negative matrix factorization for scene recognition

Accurately discriminating complicated sceneries from different categories is a useful technique in multimedia and computer vision. In this work, we propose a novel multi-view non-negative matrix factorization to detect human gaze behavior, which is subsequently integrated into an image kernel machine for scene categorization. More specifically, we first project regions from each scenery into the so-called perceptual space, which is established by combining color, texture, and semantic features. Then, a novel non-negative matrix factorization (NMF) algorithm is developed which decomposes the regions’ feature matrix into the product of the basis matrix and the sparse codes. The sparse codes indicate the saliency level of different regions which is used to constructed gaze shifting path. Thereby, the path from each scenery is derived and further incorporated into an image kernel for scene categorization. Comprehensive experiments on six scenery data sets have demonstrated the superiority of our method over a series of recognition models.

Learning community structures: Global and local perspectives

Uncovering community structures is a fundamental and important problem for analyzing complex networks. The topology information, as the direct representation of networks, is widely used for community detection. But in fact, there are other two important types of information related with network topology: the global information which captures the importance of nodes in the whole network, and the local information which describes the similarities between nodes. It is of great value to consider the information of individual nodes and information between them for community detection methods simultaneously, which is largely ignored by previous methods. In this work, we integrate the global and local information uniformly in a novel nonnegative matrix factorization (NMF) based model. Specifically, in the global aspect, we employ the PageRank to derive the importance of nodes, so that the more important the node is, the more influence the node is in the network. In the local aspect, we utilize nearness between nodes to obtain the similarities between nodes, so that nodes with larger similarities will have similar community memberships. Thereafter, we derive the multiplicative updating rule to learn the model parameter. Numerous experiments demonstrate that our approach has gained performance improvements up to almost 5% in comparison with the state-of-the-art methods.

Immersive Interactive SAR Image Representation Using Non-negative Matrix Factorization

Earth observation (EO) images clustering is a challenging problem in data mining, where each image is represented by a high-dimensional feature vector. However, the feature vectors might not be appropriate to express the semantic content of images, which eventually lead to poor results in clustering and classification. To tackle this problem, we propose an interactive approach to generate compact and informative features from images content. To this end, we utilize a 3-D interactive application to support user-images interactions. These interactions are used in the context of two novel nonnegative matrix factorization (NMF) algorithms to generate new features. We assess the quality of new features by applying k-means clustering on the generated features and compare the obtained clustering results with those achieved by original features. We perform experiments on a synthetic aperture radar (SAR) image dataset represented by different state-of-the-art features and demonstrate the effectiveness of the proposed method. Moreover, we propose a divide-and-conquer approach to cluster a massive amount of images using a small subset of interactions.

Semantic Community Identification in Large Attribute Networks

Identification of modular or community structures of a network is a key to understanding the semantics and functions of the network. While many network community detection methods have been developed, which primarily explore network topologies, they provide little semantic information of the communities discovered. Although structures and semantics are closely related, little effort has been made to discover and analyze these two essential network properties together. By integrating network topology and semantic information on nodes, e.g., node attributes, we study the problems of detection of communities and inference of their semantics simultaneously. We propose a novel nonnegative matrix factorization (NMF) model with two sets of parameters, the community membership matrix and community attribute matrix, and present efficient updating rules to evaluate the parameters with a convergence guarantee. The use of node attributes improves upon community detection and provides a semantic interpretation to the resultant network communities. Extensive experimental results on synthetic and real-world networks not only show the superior performance of the new method over the state-of-the-art approaches, but also demonstrate its ability to semantically annotate the communities.

Non‐negative matrix factorisation‐based subband decomposition for acoustic source localisation

A novel non-negative matrix factorization (NMF) based subband decomposition in frequency spatial domain for acoustic source localization using a microphone array is introduced. The proposed method decomposes source and noise subband and emphasises source dominant frequency bins for more accurate source representation. By employing NMF, delay basis vectors and their subband information in frequency spatial domain for each frame is extracted. The proposed algorithm is evaluated in both simulated noise and real noise with a speech corpus database. Experimental results clearly indicate that the algorithm performs more accurately than other conventional algorithms under both reverberant and noisy acoustic environments.

NMFE-SSCC: Non-negative matrix factorization ensemble for semi-supervised collective classification

Collective classification (CC) is a task to jointly classifying related instances of network data. Enabling CC usually improves the performance of predictive models on fully-labeled training networks with large amount of labeled data. However, acquiring such labels can be difficult and costly, and learning a CC classifier with only a few labeled data can lead to poor performance. On the other hand, there are usually large amount of unlabeled data available in practical. This naturally motivates semi-supervised collective classification (SSCC) approaches for leveraging the unlabeled data to improve CC from a sparsely-labeled network. In this paper, we propose a novel non-negative matrix factorization (NMF) based SSCC algorithm, called NMF-SSCC, to effectively learn a data representation by exploiting both labeled and unlabeled data on the network. Our idea is to use matrix factorization to obtain a compact representation of network data which uncovers the class discrimination of the data inferred from the labeled instances and simultaneously respects the intrinsic network structure. To achieve this, we design a new matrix factorization objective function and incorporate a label matrix factorization term as well as a network regularization term into it. An efficient optimization algorithm using the multiplicative updating rules is then developed to solve the new objective function. To further boost the predicting performance, we extend the proposed NMF-SSCC method into an ensemble scheme, called NMFE-SSCC, in terms of building a classification ensemble with a set of NMF-SSCC collective classifiers using different constructed latent graphs. Each NMF-SSCC classifier is learnt from one latent graph generated with various latent linkages for effectively label propagation. Experimental results on real-world data sets have demonstrated the effectiveness of the new methods.

Predicting Protein–Protein Interactions from Multimodal Biological Data Sources via Nonnegative Matrix Tri-Factorization

Protein interactions are central to all the biological processes and structural scaffolds in living organisms, because they orchestrate a number of cellular processes such as metabolic pathways and immunological recognition. Several high-throughput methods, for example, yeast two-hybrid system and mass spectrometry method, can help determine protein interactions, which, however, suffer from high false-positive rates. Moreover, many protein interactions predicted by one method are not supported by another. Therefore, computational methods are necessary and crucial to complete the interactome expeditiously. In this work, we formulate the problem of predicting protein interactions from a new mathematical perspective--sparse matrix completion, and propose a novel nonnegative matrix factorization (NMF)-based matrix completion approach to predict new protein interactions from existing protein interaction networks. Through using manifold regularization, we further develop our method to integrate different biological data sources, such as protein sequences, gene expressions, protein structure information, etc. Extensive experimental results on four species, Saccharomyces cerevisiae, Drosophila melanogaster, Homo sapiens, and Caenorhabditis elegans, have shown that our new methods outperform related state-of-the-art protein interaction prediction methods.

Smooth Nonnegative Matrix Factorization for Unsupervised Audiovisual Document Structuring

This paper introduces a new paradigm for unsupervised audiovisual document structuring. In this paradigm, a novel Nonnegative Matrix Factorization (NMF) algorithm is applied on histograms of counts (relating to a bag of features representation of the content) to jointly discover latent structuring patterns and their activations in time. Our NMF variant employs the Kullback-Leibler divergence as a cost function and imposes a temporal smoothness constraint to the activations. It is solved by a majorization-minimization technique. The approach proposed is meant to be generic and is particularly well suited to applications where the structuring patterns may overlap in time. As such, it is evaluated on two person-oriented video structuring tasks (one using the visual modality and the second the audio). This is done using a challenging database of political debate videos. Our results outperform reference results obtained by a method using Hidden Markov Models. Further, we show the potential that our general approach has for audio speaker diarization.

An NMF algorithm for blind separation of convolutive mixed source signals with least correlation constrains

Most of the existing algorithms for blind sources separation have a limitation that sources are statistically independent. However, in many practical applications, the source signals are nonnegative and mutual statistically dependent signals. When the observations are nonnegative linear combinations of nonnegative sources, the correlation coefficients of the observations are larger than these of source signals. In this letter, a novel Nonnegative Matrix Factorization (NMF) algorithm with least correlated component constraints to blind separation of convolutive mixed sources is proposed. The algorithm relaxes the source independence assumption and has low-complexity algebraic computations. Simulation results on blind source separation including real face image data indicate that the sources can be successfully recovered with the algorithm.