Semi-nonnegative Matrix Factorization Research Articles

DNSRF: Deep Network-based Semi-NMF Representation Framework

Representation learning is an important topic in machine learning, pattern recognition, and data mining research. Among many representation learning approaches, semi-nonnegative matrix factorization (SNMF) is a frequently-used one. However, a typical problem of SNMF is that usually there is no learning rate guidance during the optimization process, which often leads to a poor representation ability. To overcome this limitation, we propose a very general representation learning framework (DNSRF) that is based on deep neural net. Essentially, the parameters of the deep net used to construct the DNSRF algorithms are obtained by matrix element update. In combination with different activation functions, DNSRF can be implemented in various ways. In our experiments, we tested nine instances of our DNSRF framework on six benchmark datasets. In comparison with other state-of-the-art methods, the results demonstrate superior performance of our framework, which is thus shown to have a great representation ability.

A multi-view clustering algorithm based on deep semi-NMF

Multi-view clustering (MVC) aims to fuse the information among multiple views to achieve effective clustering. Many MVC algorithms based on semi-nonnegative matrix factorization (SNMF) typically have two issues: (1) their optimization schemes are not flexible enough; and (2) the variables are updated only rely on the data but not guided by learning rate. These problems can result in very poor clusters generated. In this paper, we present a multi-view clustering algorithm based on deep SNMF (MCDS) to resolve these issues. Specifically, we first design two types of activation functions to restrict the value domain of the element in the low-dimensional matrix to eliminate the constraint. Then, the SGD algorithm is used to implement element update guided by the learning rate. After obtaining the corresponding weight matrix and bias matrix, we combine them with the activation functions to construct a deep SNMF (DSNMF) network. This network is to update the element in the corresponding low-dimensional matrix for each view and obtain the consensus matrix. To validate the proposed algorithm, numerous experiments are performed on six multi-view datasets including both normal and large-scale datasets. The results demonstrate that MCDS can achieve excellent clustering results and outperform other competitive methods.

Distribution preserving-based deep semi-NMF for data representation

The semi-nonnegative matrix factorization (Semi-NMF) is a promising soft K-means clustering technique. Deep Semi-NMF, which stacks one-layer Semi-NMF into multi-layer, is able to learn the hierarchical projections and can obtain the deep hidden representations according to the unknown attributes of the given data. On the other hand, the inherent structure of the each data cluster can be described by the distribution of the intraclass data. Then one hopes to learn the new deep hidden representations which can preserve the intrinsic structures embedded in the original data space perfectly. Here seamlessly integrating the benefits of the Deep Semi-NMF and the distribution preserving strategy, we propose a novel distribution preserving-based deep semi-nonnegative matrix factorization method (DPNMF) to achieve this goal. In DPNMF, by maintaining the consistency of two distributions that can approximate the manifold structures, we can seek the deep hidden features which reveal the original intrinsic structures. As a result, the manifold structures in the raw data are well preserved in the new feature space. We also devise an adaptive projected Barzilai-Borwein method to optimize the proposed constrained objective function efficiently. The experimental results on the several real-world datasets show that the proposed DPNMF can achieve advantageous clustering performance in terms of accuracy (ACC), normalized mutual information (NMI), and adjusted rand index (ARI).

Latent Multi-View Semi-Nonnegative Matrix Factorization with Block Diagonal Constraint

Multi-view clustering algorithms based on matrix factorization have gained enormous development in recent years. Although these algorithms have gained impressive results, they typically neglect the spatial structures that the latent data representation should have, for example, the ideal data representation owns a block structure just like the indicator matrix has. To address this issue, a new algorithm named latent multi-view semi-nonnegative matrix factorization with block diagonal constraint (LMSNB) is proposed. First, latent representation learning and Semi-NMF are combined to get a lower-dimensional representation with consistent information from different views. Second, the block diagonal constraint is able to capture the global structure of original data. In addition, the graph regularization is considered in our model to preserve the local structure. LMSNB can deal with negative data matrix and be applied to more fields. Although the low dimensional representation from semi-nonnegative matrix factorization loses some valuable information, it still has same structure as original data with the help of block diagonal constraint and graph regularization. Finally, an iterative optimization algorithm is proposed for our objective problem. Experiments on several multi-view benchmark datasets demonstrate the effectiveness of our approach against other state-of-the-art methods.

Wide-field calcium imaging reveals widespread changes in cortical functional connectivity following mild traumatic brain injury in the mouse

>2.5 million individuals in the United States suffer mild traumatic brain injuries (mTBI) annually. Mild TBI is characterized by a brief period of altered consciousness, without objective findings of anatomic injury on clinical imaging or physical deficit on examination. Nevertheless, a subset of mTBI patients experience persistent subjective symptoms and repeated mTBI can lead to quantifiable neurological deficits, suggesting that each mTBI alters neurophysiology in a deleterious manner not detected using current clinical methods. To better understand these effects, we performed mesoscopic Ca2+ imaging in mice to evaluate how mTBI alters patterns of neuronal interactions across the dorsal cerebral cortex. Spatial Independent Component Analysis (sICA) and Localized semi-Nonnegative Matrix Factorization (LocaNMF) were used to quantify changes in cerebral functional connectivity (FC). Repetitive, mild, controlled cortical impacts induce temporary neuroinflammatory responses, characterized by increased density of microglia exhibiting de-ramified morphology. These temporary neuro-inflammatory changes were not associated with compromised cognitive performance in the Barnes maze or motor function as assessed by rotarod. However, long-term alterations in functional connectivity (FC) were observed. Widespread, bilateral changes in FC occurred immediately following impact and persisted for up to 7 weeks, the duration of the experiment. Network alterations include decreases in global efficiency, clustering coefficient, and nodal strength, thereby disrupting functional interactions and information flow throughout the dorsal cerebral cortex. A subnetwork analysis shows the largest disruptions in FC were concentrated near the impact site. Therefore, mTBI induces a transient neuroinflammation, without alterations in cognitive or motor behavior, and a reorganized cortical network evidenced by the widespread, chronic alterations in cortical FC.

Deep manifold regularized semi-nonnegative matrix factorization for Multi-view Clustering

With the development and application of modern technology, multi-view data exists widely. The deep matrix factorization model is widely used because it mines the hierarchical information of the samples, but it lacks a geometric structure protection mechanism. Moreover, in multi-view representation learning, strong representations have consensus from multi-view data and are more likely to exist in the structural relationships of data objects. To address these issues, we propose Deep Manifold Regularized Semi-Nonnegative Matrix Factorization for Multi-view Clustering (DMRMF_MVC), which preserves the geometric structure of data objects through multi-layer embedding graph regularization. The graph regularizer combines the nearest neighbor relationship with the exclusive relationship of data objects to express the internal manifold structure of the data fully. Furthermore, to distinguish the value of each view data, the adaptive weight is associated with representation learning to maximize the complementarity between the views. Extensive experimental results show that DMRMF_MVC outperforms other state-of-the-art algorithms based on multiple indicators on various datasets.

A time‐varying process monitoring approach based on iterative‐updated semi‐supervised nonnegative matrix factorizations

AbstractThis paper proposes a new time‐varying process monitoring approach based on iterative‐updated semi‐supervised nonnegative matrix factorizations (ISNMFs). ISNMFs are a type of semi‐supervised model that constructs a semi‐nonnegative matrix factorization (SNMF) model of a process using both labelled and unlabelled samples. Compared with the existing nonnegative matrix factorizations (NMFs) where NMFs are referred to as matrix factorization algorithms that factorize a nonnegative matrix into two low‐rank nonnegative matrices whose product can well approximate the original nonnegative matrix, ISNMFs have advantages in terms of the model update and the use of labelled samples. The ISNMFs‐based process monitoring approach concerns fault detection and isolation and updates an SNMF model iteratively using the latest samples to capture the change of statistical property of time‐varying processes. Moreover, the proposed fault detection and isolation approach is supported by the k‐means algorithm in theory. At last, we demonstrate the superiority of ISNMFs over the existing NMFs in terms of fault detection and isolation through a case study on the penicillin fermentation process.

Label distribution learning through exploring nonnegative components

Label distribution learning (LDL) is a new machine learning paradigm to solve label ambiguity and has drawn increasing attention in recent years. The importance of all labels needs to be considered under the LDL settings. A series of approaches have been proposed to deal with the LDL problem by considering the correlation of labels or instances. However, none of them focuses on finding interpretable bases to reduce the dimensions of the feature space. Inspired by the semi-nonnegative matrix factorization (semi-NMF) method, we propose a new LDL learning framework to deal with the problem through learning nonnegative components. The key insight is to explore the bases, each of which represents a class, through the label distribution and to transform the input matrix into a coefficient matrix of the space constructed by the bases. Consequently, a maximum entropy model can be adopted to learn the label distribution from the coefficient matrix. Experimental results on real-world datasets comparing our method with several state-of-the-art methods validate the performance of our approach.

Adaptive Graph Regularized Deep Semi-nonnegative Matrix Factorization for Data Representation

Adaptive Graph Regularized Deep Semi-nonnegative Matrix Factorization for Data Representation

Unsupervised EEG channel selection based on nonnegative matrix factorization

High-density Electroencephalogram (EEG) systems have proven to be useful in enhancing the performance of emotion recognition algorithms. However, the high-dimensional nature of this data modality may also result in irrelevant information being captured, causing overfitting problems and increasing the computational cost of downstream algorithms. To perform efficient and accurate emotion recognition, an unsupervised channel selection framework based on semi-nonnegative matrix factorization (semi-NMF) is proposed. The algorithm excels in analyzing signals with complex internal correlations and produces results that are easy to interpret. Semi-NMF was used to decompose the high-density EEG signal matrices into several activation patterns. The strongest activation pattern was considered as most related to emotion recognition and channels with large weights in that activation pattern were selected for valence-based emotion recognition. It was found that the proposed framework can effectively detect brain regions that were active during emotional activities, and, using only this reduced set of channels, achieve better recognition performance than using all channels. Compared to existing methods, the framework selects channels in a physiologically explainable way and requires no supervised feature engineering or class labels. It results in higher accuracy compared to other unsupervised energy-based methods, and on par with the supervised ReliefF method. In all, the proposed framework not only serves as a valid channel selection tool for practical emotion recognition, but also has the possibility to be transferred to other non-classification tasks, potentially contributing to a variety of EEG applications, such as brain state monitoring, pathological brain activation analysis and brain disease diagnosis.

Two-dimensional semi-nonnegative matrix factorization for clustering

In this paper, we propose a new Semi-Nonnegative Matrix Factorization method for 2-dimensional (2D) data, named TS-NMF. It overcomes the drawback of existing methods that seriously damage the spatial information of the data by converting 2D data to vectors in a preprocessing step. In particular, projection matrices are sought under the guidance of building new data representations, such that the spatial information is retained and projections are enhanced by the goal of clustering, which helps construct optimal projection directions. Moreover, to exploit nonlinear structures of the data, manifold is constructed in the projected subspace, which is adaptively updated according to the projections and less afflicted with noise and outliers of the data and thus more representative in the projected space. Hence, seeking projections, building new data representations, and learning manifold are seamlessly integrated in a single model, which mutually enhance other and lead to a powerful data representation. Comprehensive experimental results verify the effectiveness of TS-NMF in comparison with several state-of-the-art algorithms, which suggests high potential of the proposed method for real world applications.

Bi-Endmember Semi-NMF Based on Low-Rank and Sparse Matrix Decomposition

This article presents a bi-endmember semi-nonnegative matrix factorization (Semi-NMF) algorithm based on low-rank and sparse matrix decomposition (LRSMD), referred to as BLSNMF, to resolve the issues of endmember variability and nonlinear mixing. Given the fact that the hyperspectral images contain a large amount of redundant information, compressing sensing (CS) techniques can generally be used to randomly sense the effective information in an observed image according to the effective approximation of the bi-endmember components. In this article, the proposed BLSNMF integrates low-rank and sparse spaces decomposed by go decomposition (GoDec) or orthogonal subspace projection-based go decomposition (OSP-GoDec) with Semi-NMF to suppress interference between different components so as to improve the unmixing performance via a simple linear mixed model. Specifically, the observed data space is first decomposed by GoDec or OSP-GoDec to approximate four different attribute components, CS-sampled double low-rank components, structured sparse component, and noise component. Second, from the CS-sampled double low-rank components, the inherent and new endmembers along with their abundances are evaluated via Semi-NMF, and then, the double low-rank components are redescribed using the estimated endmembers and abundances. Finally, the serious interference entries in the structured sparse component space are removed from the data to better learn other attribute components. The experimental results show that BLSNMF can eliminate the interference of new endmembers and sparse noise so as to better evaluate the endmembers and abundances and effectively improve the ability to interpret the spectral information.

Discriminative deep semi-nonnegative matrix factorization network with similarity maximization for unsupervised feature learning

• We propose a DSNnet optimization model with non-convex equality constraints for unsupervised representation learning. • We design a neural network based algorithm to solve the proposed DSNnet model. • We introduce a novel similarity measurement between input data and learned features as a regularization term of DSNnet. Deep Semi-NMF (DSN), which learns hierarchical representations by stacking multiple layers Semi-NMF, shows competitive performance in unsupervised data analysis. However, the features learned from DSN always lack of representativity and discriminativity. In this paper, we build a novel Deep Semi-NMF network (DSNnet) to address the issues of DSN. Specifically, DSNnet contains multiple fully-connected layers, in which the activation function of each layer adopts Smoothly Clipped Absolute Deviation (SCAD). The non-negative hidden features are computed forwardly, while the network parameters are updated by the stochastic gradient descent method. Moreover, to enhance the discriminativity of features, we suggest simultaneously minimizing the reconstruction error of input and output, and maximizing the similarity between input and learned features. The proposed similarity measurement, which consists of global geometric similarity and local pointwise similarity, encourages the compactness between similar points and separateness between dissimilar points in the feature space, and is beneficial to preserve intrinsic information of original data. Extensive experiments conducted on several datasets illustrate the superiority of the proposed approach in comparison with state-of-the-art methods.

Autoencoder-like semi-NMF multiple clustering

Clustering is performed to partition samples into disjoint groups for facilitating the discovery of hidden patterns in the data. Many real-world applications involve various clustering methods , most of which only produce a single clustering. As a response to this issue, multiple clustering that aims to generate diverse and high-quality clustering, has emerged recently. This study proposes a novel autoencoder-like semi-nonnegative matrix factorization (NMF) multiple clustering (ASNMFMC) model that generates multiple non-redundant, high-quality clustering. The nonnegative property of the semi-NMF is utilized by the algorithm to enforce non-redundancy. Extensive experimental results demonstrate that the ASNMFMC is superior to the existing multiple clustering methods and can explore diverse high-quality clustering.

Partial multiview clustering with locality graph regularization

Multiview clustering (MVC) collects complementary and abundant information, which draws much attention in machine learning and data mining community. Existing MVC methods usually hold the assumption that all the views are complete. However, multiple source data are often incomplete in real-world applications, and so on sensor failure or unfinished collection process, which gives rise to incomplete multiview clustering (IMVC). Although enormous efforts have been devoted in IMVC, there still are some urgent issues that need to be solved: (i) The locality among multiple views has not been utilized in the existing mechanism; (ii) Existing methods inappropriately force all the views to share consensus representation while ignoring specific structures. In this paper, we propose a novel method termed partial MVC with locality graph regularization to address these issues. First, followed the traditional IMVC approaches, we construct weighted semi-nonnegative matrix factorization models to handle incomplete multiview data. Then, upon the consensus representation matrix, the locality graph is constructed for regularizing the shared feature matrix. Moreover, we add the coefficient regression term to constraint the various base matrices among views. We incorporate the three aforementioned processes into a unified framework, whereas they can negotiate with each other serving for learning tasks. An effective iterative algorithm is proposed to solve the resultant optimization problem with theoretically guaranteed convergence. The comprehensive experiment results on several benchmarks demonstrate the effectiveness of the proposed method.

Multi-View Image Clustering via Representations Fusion Method With Semi-Nonnegative Matrix Factorization

Multi-view clustering aims to obtain the perfect clusters with a set of feature sets. Many methods learn a common agreement among views to achieve this. However, they may fail to use the specific unique feature of each view, which is important to describe the intrinsic characteristics of data. Unlike many works, we treat each view as a subset feature of data and fuse representations from those unique views to learn an integrated graph for clustering. We propose a novel representations fusion method for MVC. In this method, a regularized semi-nonnegative matrix factorization is proposed to learn the low-dimensional representation of each view, in which, a regularization term is designed to keep the neighboring structure in new low-dimensional space, and it is further updated and utilized to guide the learning process. Then the learned representations is fused to get an integrated graph for describing the image data. Finally, extensive experiments on several real image datasets show that our method achieves better performance than the state-of-the-art methods.

Similarity preserving overlapping community detection in signed networks

Community detection in signed networks is a challenging research problem, and is of great importance to understanding the structural and functional properties of signed networks. It aims at dividing nodes into different clusters with more intra-cluster and less inter-cluster links. Meanwhile, most positive links should lie within clusters and most negative links should lie between clusters. In recent years, some methods for community detection in signed networks have been proposed, but few of them focus on overlapping community detection. Moreover, most of them directly exploit the sparse link topology to detect communities, which often makes them perform poorly. In view of this, in this paper we propose a similarity preserving overlapping community detection (SPOCD) method. SPOCD firstly extracts node similarity information and geometric structure information from the link topology, and then uses a graph regularized binary semi-nonnegative matrix factorization (GRBSNMF) model to fuse these two sources of information to detect communities. Through this mechanism, nodes with high similarity can be well preserved in the same community. Besides, SPOCD devises a special discretization strategy to obtain the binary community indicator matrix, which is very convenient for directly identifying overlapping communities in signed networks. We conduct extensive experiments on synthetic and real-world signed networks, and the results demonstrate that our method outperforms state-of-the-art methods.

Tight Semi-nonnegative Matrix Factorization

The nonnegative matrix factorization is a widely used, flexible matrix decomposition, finding applications in biology, image and signal processing and information retrieval, among other areas. Here we present a related matrix factorization. A multi-objective optimization problem finds conical combinations of templates that approximate a given data matrix. The templates are chosen so that as far as possible only the initial data set can be represented this way. However, the templates are not required to be nonnegative nor convex combinations of the original data.

Deep semi-nonnegative matrix factorization with elastic preserving for data representation

Deep matrix factorization methods can automatically learn the hidden representation of high dimensional data. However, they neglect the intrinsic geometric structure information of data. In this paper, we propose a Deep Semi-Nonnegative Matrix Factorization with Elastic Preserving (Deep Semi-NMF-EP) method by adding two graph regularizers in each layer. Therefore, the proposed Deep Semi-NMF-EP method effectively preserves the elasticity of data and thus can learn a better representation of high-dimensional data. In addition, we present an effective algorithm to optimize the proposed model and then provide its complexity analysis. The experimental results on the benchmark datasets show the excellent performance of our proposed method compared with other state-of-the-art methods.

An efficient ADMM-type algorithm for deep semi-nonnegative matrix factorization

In this paper, we focus on deep semi-nonnegative matrix factorization (DSemiNMF) which has a wider application in the real world than traditional NMF. We propose an efficient algorithm based on the classic alternating direction method of multipliers (ADMM) for DSemiNMF. By utilizing structures in DSemiNMF, we derive an efficient updating rule for updating subproblems according to its KKT conditions. Numerical experiments are conducted to compare the proposed algorithm with state-of-the-art deep semi-NMF algorithm. Results show that our algorithm performs better and the deep model indeed results in better clustering accuracy than single-layer model.