Semi-supervised Non-negative Matrix Factorization Research Articles

Clustering Network Traffic Using Semi-Supervised Learning

Clustering algorithms play a crucial role in early warning cybersecurity systems. They allow for the detection of new attack patterns and anomalies and enhance system performance. This paper discusses the problem of clustering data collected by a distributed system of network honeypots. In the proposed approach, when a network flow matches an attack signature, an appropriate label is assigned to it. This enables the use of semi-supervised learning algorithms and improves the quality of clustering results. The article compares the results of learning algorithms conducted with and without partial supervision, particularly non-negative matrix factorization and semi-supervised non-negative matrix factorization. Our results confirm the positive impact of labeling a portion of flows on the quality of clustering.

Semi-supervised Multi-view Clustering based on NMF with Fusion Regularization

Multi-view clustering has attracted significant attention and application. Nonnegative matrix factorization is one popular feature of learning technology in pattern recognition. In recent years, many semi-supervised nonnegative matrix factorization algorithms were proposed by considering label information, which has achieved outstanding performance for multi-view clustering. However, most of these existing methods have either failed to consider discriminative information effectively or included too much hyper-parameters. Addressing these issues, a semi-supervised multi-view nonnegative matrix factorization with a novel fusion regularization (FRSMNMF) is developed in this article. In this work, we uniformly constrain alignment of multiple views and discriminative information among clusters with designed fusion regularization. Meanwhile, to align the multiple views effectively, two kinds of compensating matrices are used to normalize the feature scales of different views. Additionally, we preserve the geometry structure information of labeled and unlabeled samples by introducing the graph regularization simultaneously. Due to the proposed methods, two effective optimization strategies based on multiplicative update rules are designed. Experiments implemented on six real-world datasets have demonstrated the effectiveness of our FRSMNMF comparing with several state-of-the-art unsupervised and semi-supervised approaches.

Semi-supervised nonnegative matrix factorization with label propagation and constraint propagation

Semi-supervised nonnegative matrix factorization (SNMF) improves the clustering effect of nonnegative matrix factorization (NMF) by integrating label information. However, as one of the most commonly used semi-supervised learning methods, label propagation algorithm (LP) has some limitations due to its heavy dependence on the predefined similarity matrix. To address this deficiency and effectively use limited supervisory information, a structural NMF algorithm with label propagation and constraint propagation (PNLP-SCNMF) is proposed in this paper, which simultaneously performs data representation and classification in a joint manner. Both positive label information and negative label information are introduced into the proposed method, and the factor matrix is constrained by label information to construct the tri-decomposition form. Constraint propagation algorithm (CPA) is used to correct the constructed similarity matrix, so that the low-dimensional subspace obtained by matrix decomposition can retain the geometric structure characteristics of the original sample space. In addition, an efficient iterative update optimization scheme is proposed to solve this objective function, and its convergence is proved theoretically. A large number of comparative experiments on nine real datasets show that the proposed method can consistently achieve higher clustering accuracy in image clustering tasks.

Dual-Graph-Regularization Constrained Nonnegative Matrix Factorization with Label Discrimination for Data Clustering

NONNEGATIVE matrix factorization (NMF) is an effective technique for dimensionality reduction of high-dimensional data for tasks such as machine learning and data visualization. However, for practical clustering tasks, traditional NMF ignores the manifold information of both the data space and feature space, as well as the discriminative information of the data. In this paper, we propose a semisupervised NMF called dual-graph-regularization-constrained nonnegative matrix factorization with label discrimination (DCNMFLD). DCNMFLD combines dual graph regularization and prior label information as additional constraints, making full use of the intrinsic geometric and discriminative structures of the data, and can efficiently enhance the discriminative and exclusionary nature of clustering and improve the clustering performance. The evaluation of the clustering experimental results on four benchmark datasets demonstrates the effectiveness of our new algorithm.

Inter- and intra-hypergraph regularized nonnegative matrix factorization with hybrid constraints

The accurate low rank representation of high-dimensional data learned by the manifold regularized nonnegative matrix factorization framework is effective in data clustering. In previous work, this work has mainly been solved in the way of similarity matrix induction. To further increase the efficacy of low-rank representations, we propose a novel semi-supervised non-negative matrix factorization (NMF) model in this study called inter- and intra-hypergraph regularized non-negative matrix factorization with hybrid constraints (IGNMFC). IGNMFC constructs intra-hypergraph regularization and intra-hypergraph regularization by hypergraph learning, which can precisely induce high-dimensional data to map toward low-dimensional. Moreover, hybrid constraints are introduced to improve the exclusivity and sparsity of low-dimensional representations, and the result accounts for the benefit of this method in learning distinguishable subspace representations. Finally, IGNMFC is transformed into an optimal problem and an efficient iteration rule is proposed. Experiments on several datasets demonstrate that the proposed method outperforms the state-of-the-art NMF algorithms, and can achieve at least 7.9%∼15% accuracy improvement in most cases.

Incremental semi-supervised graph learning NMF with block-diagonal

Non-negative matrix factorization (NMF), as a good data dimensionality reduction method, is widely used in the field of image recognition. Incremental non-negative matrix factorization (INMF) as an improvement solves the problem of inefficiency caused by repeated running of data samples during online learning. However, in the traditional incremental non-negative matrix factorization algorithm, the newly added training samples do not contain label information. Some samples may be marked in both the initial sample and the new training sample in the real application scenario. In order to make full use of the label information carried by the dataset, in this paper, we propose a semi-supervised non-negative matrix factorization model for batch incremental data, incremental semi-supervised graph learning NMF with block diagonal (ISGDNMF). The model is divided into three cases according to the degree of label-carrying for the new batch data: all label-carrying, no label-carrying, and partial label-carrying. The label information is also used to add a diagonal structure to the coefficient matrix, which makes it possible to have stronger discriminatory ability and to distinguish different classes of images more easily. And graph regularization information is added in order to maintain the spatial-geometric structure of the data. Experiments on six image datasets show that this algorithm has superior performance relative to the other seven NMF-based algorithms.

Multiview Clustering via Hypergraph Induced Semi-Supervised Symmetric Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) based multiview technique has been commonly used in multiview data clustering tasks. However, previous NMF based multiview clustering approaches fail to take advantage of a small amount of supervisory information to effectively improve the clustering performance, and are easily affected by the additional post-processing method in clustering tasks. To cope with these issues, a novel framework named multiview clustering via hypergraph induced semi-supervised symmetric NMF (MVCHSS) is proposed in this paper for multiview data clustering applications. Specifically, the proposed method has the following features: 1) a new multiview based hypergraph pairwise constraints propagation (MHPCP) algorithm is developed in MVCHSS to construct a set of informative similarity matrices, revealing the high-order relationships effectively and fully utilizing the limited pairwise constraint supervisory information among samples of each view data; 2) the obtained similarity matrices with much supervisory information are not only enforced into the symmetric NMF (SNMF) model, but also incorporated into the graph regularization for each view data; 3) the optimization problem of MVCHSS is formulated for multiview data clustering tasks to acquire a more discriminative clustering indicator matrix (or called consensus assignment matrix) without additional post-processing method. Moreover, the proof of convergence and the computational complexity for MVCHSS are presented. Extensive experiments on five multiview datasets demonstrate that the proposed MVCHSS framework outperforms several state-of-the-art multiview clustering methods.

Dual Space Latent Representation Learning for Image Representation

Semi-supervised non-negative matrix factorization (NMF) has achieved successful results due to the significant ability of image recognition by a small quantity of labeled information. However, there still exist problems to be solved such as the interconnection information not being fully explored and the inevitable mixed noise in the data, which deteriorates the performance of these methods. To circumvent this problem, we propose a novel semi-supervised method named DLRGNMF. Firstly, dual latent space is characterized by the affinity matrix to explicitly reflect the interrelationship between data instances and feature variables, which can exploit the global interconnection information in dual space and reduce the adverse impacts caused by noise and redundant information. Secondly, we embed the manifold regularization mechanism in the dual graph to steadily retain the local manifold structure of dual space. Moreover, the sparsity and the biorthogonal condition are integrated to constrain matrix factorization, which can greatly improve the algorithm’s accuracy and robustness. Lastly, an effective alternating iterative updating method is proposed, and the model is optimized. Empirical evaluation on nine benchmark datasets demonstrates that DLRGNMF is more effective than competitive methods.

Quantification of subtype purity in Luminal A breast cancer predicts clinical characteristics and survival

PurposePAM50 profiling assigns each breast cancer to a single intrinsic subtype based on a bulk tissue sample. However, individual cancers may show evidence of admixture with an alternate subtype that could affect prognosis and treatment response. We developed a method to model subtype admixture using whole transcriptome data and associated it with tumor, molecular, and survival characteristics for Luminal A (LumA) samples.MethodsWe combined TCGA and METABRIC cohorts and obtained transcriptome, molecular, and clinical data, which yielded 11,379 gene transcripts in common and 1,178 cases assigned to LumA. We used semi-supervised non-negative matrix factorization (ssNMF) to compute the subtype admixture proportions of the four major subtypes—pLumA, pLumB, pHER2, and pBasal—for each case and measured associations with tumor characteristics, molecular features, and survival.ResultsLuminal A cases in the lowest versus highest quartile for pLumA transcriptomic proportion had a 27% higher prevalence of stage > 1, nearly a threefold higher prevalence of TP53 mutation, and a hazard ratio of 2.08 for overall mortality. We found positive associations between pHER2 and HER2 positivity by IHC or FISH; between pLumB and PR negativity; and between pBasal and younger age, node positivity, TP53 mutation, and EGFR expression. Predominant basal admixture, in contrast to predominant LumB or HER2 admixture, was not associated with shorter survival.ConclusionBulk sampling for genomic analyses provides an opportunity to expose intratumor heterogeneity, as reflected by subtype admixture. Our results elucidate the striking extent of diversity among LumA cancers and suggest that determining the extent and type of admixture holds promise for refining individualized therapy. LumA cancers with a high degree of basal admixture appear to have distinct biological characteristics that warrant further study.

SSNMDI: a novel joint learning model of semi-supervised non-negative matrix factorization and data imputation for clustering of single-cell RNA-seq data.

Single-cell RNA sequencing (scRNA-seq) technology attracts extensive attention in the biomedical field. It can be used to measure gene expression and analyze the transcriptome at the single-cell level, enabling the identification of cell types based on unsupervised clustering. Data imputation and dimension reduction are conducted before clustering because scRNA-seq has a high 'dropout' rate, noise and linear inseparability. However, independence of dimension reduction, imputation and clustering cannot fully characterize the pattern of the scRNA-seq data, resulting in poor clustering performance. Herein, we propose a novel and accurate algorithm, SSNMDI, that utilizes a joint learning approach to simultaneously perform imputation, dimensionality reduction and cell clustering in a non-negative matrix factorization (NMF) framework. In addition, we integrate the cell annotation as prior information, then transform the joint learning into a semi-supervised NMF model. Through experiments on 14 datasets, we demonstrate that SSNMDI has a faster convergence speed, better dimensionality reduction performance and a more accurate cell clustering performance than previous methods, providing an accurate and robust strategy for analyzing scRNA-seq data. Biological analysis are also conducted to validate the biological significance of our method, including pseudotime analysis, gene ontology and survival analysis. We believe that we are among the first to introduce imputation, partial label information, dimension reduction and clustering to the single-cell field. The source code for SSNMDI is available at https://github.com/yushanqiu/SSNMDI.

Continuous Semi-Supervised Nonnegative Matrix Factorization

Nonnegative matrix factorization can be used to automatically detect topics within a corpus in an unsupervised fashion. The technique amounts to an approximation of a nonnegative matrix as the product of two nonnegative matrices of lower rank. In certain applications it is desirable to extract topics and use them to predict quantitative outcomes. In this paper, we show Nonnegative Matrix Factorization can be combined with regression on a continuous response variable by minimizing a penalty function that adds a weighted regression error to a matrix factorization error. We show theoretically that as the weighting increases, the regression error in training decreases weakly. We test our method on synthetic data and real data coming from Rate My Professors reviews to predict an instructor’s rating from the text in their reviews. In practice, when used as a dimensionality reduction method (when the number of topics chosen in the model is fewer than the true number of topics), the method performs better than doing regression after topics are identified—both during training and testing—and it retrains interpretability.

Phonocardiogram signal classification for the detection of heart valve diseases using robust conglomerated models

The diagnosis of cardiovascular diseases is quite important in the field of medical community. An important physiological signal of human body is heart sound and it arises due to the blood turbulence and pulsing of cardiac structures. For the early diagnosis of heart diseases, the analysis of heart sounds play an important role as they contain a huge quantity of pathological information associated with heart. To detect heart sounds, Phonocardiogram (PCG) is used as it is a highly useful and non-invasive technique and can be easily analyzed well. In this paper, some efficient models are proposed for the classification of PCG signals. Two important and robust conglomerated models are proposed initially, wherein the first strategy utilizes the concept of semi-supervised Non-negative Matrix Factorization (NMF) along with Brain Storming (BS) optimization algorithm and an advanced version of BS termed as Advanced BS (ABS) is proposed and then it is merged with Genetic Programming (GP) so that new algorithms such as BS-GP and ABS-GP are formed and finally the features selected through it are fed to classification through machine learning. The second strategy utilizes the concept of using three dimensionality reduction techniques along with Fuzzy C-means (FCM) clustering and then an Advanced Sine-Cosine (ASC) optimization algorithm with three different modifications is proposed for the purpose of feature selection and finally it is classified. Deep learning techniques were also employed in the study such as the usage of an Attention based Bidirectional Long Short-Term Memory (A-BLSTM), Ordinal Variational Autoencoder (O-VAE), Conditional Variational Autoencoders (CVAE), Hyperspherical CVAE (H-CVAE) and the Restricted Boltzmann Machine based Deep Belief Network (RBM-DBN) for the classification of PCG signals. The experiment is conducted on a publicly available dataset and results show that a high classification accuracy of 95.39% is obtained for the semi-supervised NMF concept with ABS-GP technique and Support Vector Machine (SVM) classifier.

Self-supervised semi-supervised nonnegative matrix factorization for data clustering

Semi-supervised nonnegative matrix factorization exploits the strengths of matrix factorization in successfully learning part-based representation and is also able to achieve high learning performance when facing a scarcity of labeled data and a large amount of unlabeled data. Its major challenge lies in how to learn more discriminative representations from limited labeled data. Furthermore, self-supervised learning has been proven very effective at learning representations from unlabeled data in various learning tasks. Recent research works focus on utilizing the capacity of self-supervised learning to enhance semi-supervised learning. In this paper, we design an effective Self-Supervised Semi-Supervised Nonnegative Matrix Factorization (S4NMF) in a semi-supervised clustering setting. The S4NMF directly extracts a consensus result from ensembled NMFs with similarity and dissimilarity regularizations. In an iterative process, this self-supervisory information will be fed back to the proposed model to boost semi-supervised learning and form more distinct clusters. The proposed iterative algorithm is used to solve the given problem, which is defined as an optimization problem with a well-formulated objective function. In addition, the theoretical and empirical analyses investigate the convergence of the proposed optimization algorithm. To demonstrate the effectiveness of the proposed model in semi-supervised clustering, we conduct extensive experiments on standard benchmark datasets. The source code for reproducing our results can be found at https://github.com/ChavoshiNejad/S4NMF.

Multiple Graph Adaptive Regularized Semi-Supervised Nonnegative Matrix Factorization with Sparse Constraint for Data Representation

Multiple graph and semi-supervision techniques have been successfully introduced into the nonnegative matrix factorization (NMF) model for taking full advantage of the manifold structure and priori information of data to capture excellent low-dimensional data representation. However, the existing methods do not consider the sparse constraint, which can enhance the local learning ability and improve the performance in practical applications. To overcome this limitation, a novel NMF-based data representation method, namely, the multiple graph adaptive regularized semi-supervised nonnegative matrix factorization with sparse constraint (MSNMFSC) is developed in this paper for obtaining the sparse and discriminative data representation and increasing the quality of decomposition of NMF. Particularly, based on the standard NMF, the proposed MSNMFSC method combines the multiple graph adaptive regularization, the limited supervised information and the sparse constraint together to learn the more discriminative parts-based data representation. Moreover, the convergence analysis of MSNMFSC is studied. Experiments are conducted on several practical image datasets in clustering tasks, and the clustering results have shown that MSNMFSC achieves better performance than several most related NMF-based methods.

Label consistency-based deep semisupervised NMF for tumor recognition

Tumor has become a hot topic in the field of image processing and pattern recognition. Gene expression data is an important way to study tumor. Since gene expression data is characterized by high-dimensional small samples, it is one of the key steps to extract discriminative gene features to distinguish different tumor types. Nonnegative matrix factorization (NMF) is an unsupervised feature representation method that does not depend on the label information of data. NMF can achieve nonlinear dimension reduction and is widely used in tumor recognition. Considering that some data may carry label information, models’ feature representation capability will be improved if the label information can be used effectively. Therefore, this paper proposes a semisupervised NMF model based on label consistency (LC-NMF), which uses both labeled and unlabeled data to obtain feature representations. Furthermore, to alleviate the sensitivity of the NMF model to initial values and excavate the deep features of data, a label consistency-based deep semisupervised NMF model (LC-DNMF) is constructed, which combines the LC-NMF model with the layer-by-layer pretraining and multilayer representation strategy in deep learning. The performance of the proposed models (i.e., LC-NMF and LC-DNMF) is verified by applying them to the tumor recognition tasks. The experimental results on seven datasets show that the two models achieve good results and can obtain competitive recognition accuracies compared with the state-of-the-art methods. Furthermore, the performance of the LC-DNMF model outperforms that of the LC-NMF model, which verifies the effectiveness of introducing the layer-by-layer pretraining and multilayer representation strategy.

Semi-supervised nonnegative matrix factorization with pairwise constraints for image clustering

Semi-supervised nonnegative matrix factorization with pairwise constraints for image clustering

Semi-supervised dual-graph regularization non-negative matrix factorization with local coordinate and orthogonal constraints for image clustering

Due to significant performance in the representation of data points, non-negative matrix factorization (NMF) has been widely applied in machine-learning fields, such as dimension reduction, image representation, feature extraction, data mining, and so on. However, classical NMF suffers from a common issue, low efficiency in representing the internal geometric structure of data and sparsity limitation. To circumvent this problem, we innovatively propose a semi-supervised NMF algorithm called semi-supervised dual-graph regularization non-negative matrix factorization (LOSDNMF), into which dual-graph and bi-orthogonal constraints are embedded to reduce the inconsistency between the original matrix and the basic vectors while maintaining the manifold structures of the data and feature spaces. This strategy can fully explore the potential geometry information of the data, which is extremely beneficial to enhance the learning ability of the model. In addition, the local coordinate constraints are introduced to ensure good sparsity of the coefficient matrix and simplify the calculation. Furthermore, an iterative updating scheme for the optimization problem of LOSDNMF and its convergence proofs are also provided in detail. The effectiveness of the proposed method is verified on eight benchmark datasets. Experimental results show that our method can effectively improve clustering performance.

Differentiated graph regularized non-negative matrix factorization for semi-supervised community detection

Semi-supervised non-negative matrix factorization (Semi-NMF) has been widely used in community detection by employing the side information. However, the graph used in previous Semi-NMF methods only takes into account single graph construction, being aware of specific similarity measurements among the community nodes. In this paper, we propose a novel approach, named Differentiated Graph regularized Non-negative Matrix Factorization (DGNMF), for semi-supervised community detection by leveraging the paired constraints between network nodes. In particular, the similarity and dissimilarity constraints on must-link and cannot-link data samples are imposed respectively to construct a differentiated graph that is involved into the proposed algorithm in a form of dual-sparse NMF problem. To solve the optimization problem, we propose an alternate iterative algorithm and demonstrate its convergence theoretically. Extensive experiments on two artificial networks and ten real-world networks show that the proposed DGNMF can effectively improve the accuracy of community detection compared with the state-of-the-art NMF-based approaches.

Semi-supervised sparse neighbor constrained co-clustering with dissimilarity and similarity regularization

Nonnegative matrix factorization (NMF) is a very effective method for high dimensional data analysis, which has been widely used in computer vision. However, the conventional NMF is unsupervised, and thus, it cannot utilize the label information. To this end, semi-supervised NMF is proposed, which performs the NMF with the guidance of the supervisory information. However, semi-supervised NMF fails to make full use of label information, which limits the performance of clustering using low dimensional representation, and samples and features cannot be clustered in a mutually reinforcing manner. To solve the above shortcomings, we propose a semi-supervised sparse neighbor constrained co-clustering with dissimilarity and similarity regularization model (SSCCDS). First, co-clustering is introduced to cluster samples and features simultaneously. Secondly, the model imposes similarity and dissimilarity regularization constraints on samples by low-dimensional representations. Specifically, similarity and dissimilar regularization constraints are imposed on labeled samples, and similarity regularization constraints are imposed on unlabeled data. Thirdly, the model proposes sparse neighbor constraints for feature consistent learning. Then, a multiplicative alternating scheme is proposed for objective optimization. A large number of experiments on different data sets show that SSCCDS has good clustering ability.

Guided Semi-Supervised Non-Negative Matrix Factorization

Classification and topic modeling are popular techniques in machine learning that extract information from large-scale datasets. By incorporating a priori information such as labels or important features, methods have been developed to perform classification and topic modeling tasks; however, most methods that can perform both do not allow for guidance of the topics or features. In this paper, we propose a novel method, namely Guided Semi-Supervised Non-negative Matrix Factorization (GSSNMF), that performs both classification and topic modeling by incorporating supervision from both pre-assigned document class labels and user-designed seed words. We test the performance of this method on legal documents provided by the California Innocence Project and the 20 Newsgroups dataset. Our results show that the proposed method improves both classification accuracy and topic coherence in comparison to past methods such as Semi-Supervised Non-negative Matrix Factorization (SSNMF), Guided Non-negative Matrix Factorization (Guided NMF), and Topic Supervised NMF.