Convex Nonnegative Matrix Factorization Research Articles

Identification of liquor adulteration by Raman spectroscopy method based on ICNAFS

The health of consumers can be impacted by the additives placed into the liquor. To address the issues of poor accuracy, low reliability, and complex operational procedures in identifying adulteration in existing liquor, an improved convex non-negative matrix factorization (ICNAFS) with an adaptive graph constraint for unsupervised feature extraction is proposed in this paper, with the goal of achieving rapid identification of adulteration in liquor by Raman spectroscopy through dimensionality reduction. For the sake to streamline the calculation process for effective feature extraction and increase the accuracy of the analyzed model, the proposed ICNAFS method incorporates two fundamental models, such as ridge regression and convex non-negative matrix factorization (NMF). In particular, dimensionality reduction of the original spectrum is initially conducted using Principal Component Analysis (PCA), Sequential Projection Algorithm (SPA), Convex Non-Negative Matrix Factorization with an Adaptive Graph Constraint (CNAFS), and ICNAFS respectively. k-means is subsequently employed to merge the four models for clustering analysis. The results suggest that the accuracy of the presented ICNAFS-assisted k-means model is higher than the other techniques, with a clustering accuracy of 98.67%, exhibiting a 4% improvement over the existing CNAFS, through examination of 150 sets of tainted liquor data from five categories of samples. This demonstrates the potency of the proposed ICNAFS-assisted k-means clustering model in conjunction with Raman spectroscopy as a method for detecting tainted liquor.

Global and local similarity learning in multi-kernel space for nonnegative matrix factorization

Most of existing nonnegative matrix factorization (NMF) methods do not fully exploit global and local similarity information from data. In this paper, we propose a novel local similarity learning approach in the convex NMF framework, which encourages inter-class separability that is desired for clustering. Thus, the new model is capable of enhancing intra-class similarity and inter-class separability with simultaneous global and local learning. Moreover, the model learns the factor matrices in an augmented kernel space, which is a convex combination of pre-defined kernels with auto-learned weights. Thus, the learnings of cluster structure, representation factor matrix, and the optimal kernel mutually enhance each other in a seamlessly integrated model, which leads to informative representation. Multiplicative updating rules are developed with theoretical convergence guarantee. Extensive experimental results have confirmed the effectiveness of the proposed model.

A comparison of non-negative matrix underapproximation methods for the decomposition of magnetic resonance spectroscopy data from human brain tumors.

Magnetic resonance spectroscopy (MRS) is an MR technique that provides information about the biochemistry of tissues in a noninvasive way. MRS has been widely used for the study of brain tumors, both preoperatively and during follow-up. In this study, we investigated the performance of a range of variants of unsupervised matrix factorization methods of the non-negative matrix underapproximation (NMU) family, namely, sparse NMU, global NMU, and recursive NMU, and compared them with convex non-negative matrix factorization (C-NMF), which has previously shown a good performance on brain tumor diagnostic support problems using MRS data. The purpose of the investigation was 2-fold: first, to ascertain the differences among the sources extracted by these methods; and second, to compare the influence of each method in the diagnostic accuracy of the classification of brain tumors, using them as feature extractors. We discovered that, first, NMU variants found meaningful sources in terms of biological interpretability, but representing parts of the spectrum, in contrast to C-NMF; and second, that NMU methods achieved better classification accuracy than C-NMF for the classification tasks when one class was not meningioma.

Risk Budget Portfolios With Convex Non-negative Matrix Factorization

Risk Budget Portfolios With Convex Non-negative Matrix Factorization

Convex Non-Negative Matrix Factorization With Adaptive Graph for Unsupervised Feature Selection.

Unsupervised feature selection (UFS) aims to remove the redundant information and select the most representative feature subset from the original data, so it occupies a core position for high-dimensional data preprocessing. Many proposed approaches use self-expression to explore the correlation between the data samples or use pseudolabel matrix learning to learn the mapping between the data and labels. Furthermore, the existing methods have tried to add constraints to either of these two modules to reduce the redundancy, but no prior literature embeds them into a joint model to select the most representative features by the computed top ranking scores. To address the aforementioned issue, this article presents a novel UFS method via a convex non-negative matrix factorization with an adaptive graph constraint (CNAFS). Through convex matrix factorization with adaptive graph constraint, it can dig up the correlation between the data and keep the local manifold structure of the data. To our knowledge, it is the first work that integrates pseudo label matrix learning into the self-expression module and optimizes them simultaneously for the UFS solution. Besides, two different manifold regularizations are constructed for the pseudolabel matrix and the encoding matrix to keep the local geometrical structure. Eventually, extensive experiments on the benchmark datasets are conducted to prove the effectiveness of our method. The source code is available at: https://github.com/misteru/CNAFS.

Does non-linear factorization of financial returns help build better and stabler portfolios?

A portfolio allocation method based on linear and non-linear latent constrained conditional factors is presented. The factor loadings are constrained to always be positive in order to obtain long-only portfolios, which is not guaranteed by classical factor analysis or PCA. In addition, the factors are to be uncorrelated among clusters in order to build long-only portfolios. Our approach is based on modern machine learning tools: convex Non-negative Matrix Factorization (NMF) and autoencoder neural networks, designed in a specific manner to enforce the learning of useful hidden data structure such as correlation between the assets' returns. Our technique finds lowly correlated linear and non-linear conditional latent factors which are used to build outperforming global portfolios consisting of cryptocurrencies and traditional assets, similar to hierarchical clustering method. We study the dynamics of the derived non-linear factors in order to forecast tail losses of the portfolios and thus build more stable ones.

Memory-Efficient Convex Optimization for Self-Dictionary Separable Nonnegative Matrix Factorization: A Frank–Wolfe Approach

Nonnegative matrix factorization (NMF) often relies on the separability condition for tractable algorithm design. Separability-based NMF is mainly handled by two types of approaches, namely, greedy pursuit and convex programming. A notable convex NMF formulation is the so-called self-dictionary multiple measurement vectors (SD-MMV), which can work without knowing the matrix rank a priori, and is arguably more resilient to error propagation relative to greedy pursuit. However, convex SD-MMV renders a large memory cost that scales quadratically with the problem size. This memory challenge has been around for a decade, and a major obstacle for applying convex SD-MMV to big data analytics. This work proposes a memory-efficient algorithm for convex SD-MMV. Our algorithm capitalizes on the special update rules of a classic algorithm from the 1950s, namely, the Frank-Wolfe (FW) algorithm. It is shown that, under reasonable conditions, the FW algorithm solves the noisy SD-MMV problem with a memory cost that grows linearly with the amount of data. To handle noisier scenarios, a smoothed group sparsity regularizer is proposed to improve robustness while maintaining the low memory footprint with guarantees. The proposed approach presents the first linear memory complexity algorithmic framework for convex SD-MMV based NMF. The method is tested over a couple of unsupervised learning tasks, i.e., text mining and community detection, to showcase its effectiveness and memory efficiency.

Dual semi-supervised convex nonnegative matrix factorization for data representation

Semi-supervised nonnegative matrix factorization (NMF) has received considerable attention in machine learning and data mining. A new semi-supervised NMF method, called dual semi-supervised convex nonnegative matrix factorization (DCNMF), is proposed in this paper for fully using the limited label information. Specifically, DCNMF simultaneously incorporates the pointwise and pairwise constraints of labeled samples as dual supervisory information into convex NMF, which results in a better low-dimensional data representation. Moreover, DCNMF imposes the nonnegative constraint only on the coefficient matrix but not on the base matrix. Consequently, DCNMF can process mixed-sign data, and hence enlarge the range of applications. We derive an efficient alternating iterative algorithm for DCNMF to solve the optimization, and analyze the proposed DCNMF method in terms of the convergence and computational complexity. We also discuss the relationships between DCNMF and several typical NMF based methods. Experimental results illustrate that DCNMF outperforms the related state-of-the-art NMF methods on nonnegative and mixed-sign datasets for clustering applications.

A Regularized Convex Nonnegative Matrix Factorization Model for signed network analysis

Community detection and link prediction are two basic tasks of complex network system analysis, which are widely used in the detection of telecom fraud organizations and recommendation systems in the real world. In ordinary unsigned networks, these two analyses have been developed for a long time. However, due to the existence of negative edges, the study of community detection and link prediction in signed networks is still limited now. Most existing methods have high computational complexity and ignore the generation of the networks based on heuristics. In this paper, we propose a regularized convex nonnegative matrix factorization model (RC-NMF) from the perspective of the generative model to detection communities in the signed network. This algorithm introduces graph regularization to constrain nodes with negative edges into different communities and nodes with positive edges into the same communities as much as possible. Experiments on synthetic signed networks and several real-world signed networks validate the effectiveness and accuracy of the proposed approach both in community detection and link prediction.

Robust Structured Convex Nonnegative Matrix Factorization for Data Representation

Nonnegative Matrix Factorization (NMF) is a popular technique for machine learning. Its power is that it can decompose a nonnegative matrix into two nonnegative factors whose product well approximates the nonnegative matrix. However, the nonnegative constraint of the data matrix limits its application. Additionally, the representations learned by NMF fail to respect the intrinsic geometric structure of the data. In this paper, we propose a novel unsupervised matrix factorization method, called Robust Structured Convex Nonnegative Matrix Factorization (RSCNMF). RSCNMF not only achieves meaningful factorizations of the mixed-sign data, but also learns a discriminative representation by leveraging local and global structures of the data. Moreover, it introduces the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm loss function to deal with noise and outliers, and exploits the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm feature regularizer to select discriminative features across all the samples. We develop an alternate iterative scheme to solve such a new model. The convergence of RSCNMF is proven theoretically and verified empirically. The experimental results on eight real-world data sets show that our RSCNMF algorithm matches or outperforms the state-of-the-art methods.

Total Variation Constrained Graph-Regularized Convex Non-Negative Matrix Factorization for Data Representation

We propose a novel NMF algorithm, named Total Variation constrained Graph-regularized Convex Non-negative Matrix Factorization (TV-GCNMF), to incorporate total variation and graph Laplacian with convex NMF. In this model, the feature details of the data are preserved by a diffusion coefficient based on the gradient information. The graph regularization and convex constraints reveal the intrinsic geometry and structure information of the features; thereby, obtaining sparse and parts-based representations. Furthermore, we give the multiplicative update rules and prove convergence of the proposed algorithm. The results of clustering experiments on multiple datasets, under various noise conditions, show the effectiveness and robustness of the proposed method compared to state-of-the-art clustering methods and other related work.

FER based on the improved convex nonnegative matrix factorization feature

Facial expression recognition is an important research issue in the pattern recognition field. In this paper, we intend to present an accurate facial expression recognition (FER) system, which employs an improved convex non-negative matrix factorization (ICNMF) method based on a novel objective function and smaller iterative step sizes for feature extraction. Since negative values appearing in the facial expression feature will weaken the features and reduce the recognition rate, the nonnegative matrix factorization (NMF) methods are adopted to guarantee the non-negativity of the extracted feature value to improve the recognition rate. To enhance the performance of NMF method for FER, the ICNMF approach based on a novel convergent objective function and smaller iterative step sizes is proposed, and the FER rate can be improved effectively. In the FER system, the face region is detected firstly, and is enhanced by histogram specification, secondly the ICNMF approach is adopted to extract features and then the feature coefficient matrix is achieved. Finally, the SVM classifier is applied to recognize the extracted features. To validate the effectiveness of FER system, four public available datasets of MultiPIE, CK+, FER2013 and SFEW are tested and then high recognition rates can be achieved based on ICNMF method. In addition, the proposed ICNMF approach is compared with the methods of multi-layer NMF, sparse non-negative matrix factorization (SNMF), the traditional convex non-negative matrix factorization (CNMF), deep belief networks (DBN) and stacked auto-encoder (SAE), and results of experiments show that the proposed ICNMF approach is significantly effective contrasting to the other five expression extraction methods.

Near-Convex Archetypal Analysis

Nonnegative matrix factorization (NMF) is a widely used linear dimensionality reduction technique for nonnegative data. NMF requires that each data point is approximated by a convex combination of basis elements. Archetypal analysis (AA), also referred to as convex NMF, is a well-known NMF variant imposing that the basis elements are themselves convex combinations of the data points. AA has the advantage to be more interpretable than NMF because the basis elements are directly constructed from the data points. However, it usually suffers from a high data fitting error because the basis elements are constrained to be contained in the convex cone of the data points. In this letter, we introduce near-convex archetypal analysis (NCAA) which combines the advantages of both AA and NMF. As for AA, the basis vectors are required to be linear combinations of the data points and hence are easily interpretable. As for NMF, the additional flexibility in choosing the basis elements allows NCAA to have a low data fitting error. We show that NCAA compares favorably with a state-of-the-art minimum-volume NMF method on synthetic datasets and on a real-world hyperspectral image.

A New Image-based Safety Monitoring Technique with Applications to a Steel Coil Painting Process

This article develops a brand new technique to perform safety monitoring using images for a type of complex industrial processes. The new safety monitoring technique is developed with the aid of a modified nonnegative matrix factorization (NMF) algorithm that is called multiple-centroid strictly convex nonnegative matrix factorization (MCSCNMF). Specifically, MCSCNMF is designed deliberately in an all-new manner such that it can learn more accurate centroids for each group of samples than the existing NMF-like algorithms. Moreover, MCSCNMF takes advantage of multiple centroids instead of single centroid to describe the complicated distribution of each type of samples. Both properties give MCSCNMF a powerful clustering performance and make it beneficial for developing high-performance safety monitoring techniques than the existing NMF-like algorithms. We compare different types of monitoring methods in terms of false negative, false positive, and average approximation error in an experiment on a steel coil painting process. Comparison results demonstrate that the MCSCNMF-based monitoring technique can obtain the best experimental results compared with other monitoring methods all the time.

Extraction of artefactual MRS patterns from a large database using non-negative matrix factorization.

Despite the success of automated pattern recognition methods in problems of human brain tumor diagnostic classification, limited attention has been paid to the issue of automated data quality assessment in the field of MRS for neuro-oncology. Beyond some early attempts to address this issue, the current standard in practice is MRS quality control through human (expert-based) assessment. One aspect of automatic quality control is the problem of detecting artefacts in MRS data. Artefacts, whose variety has already been reviewed in some detail and some of which may even escape human quality control, have a negative influence in pattern recognition methods attempting to assist tumor characterization. The automatic detection of MRS artefacts should be beneficial for radiology as it guarantees more reliable tumor characterizations, as well as the development of more robust pattern recognition-based tumor classifiers and more trustable MRS data processing and analysis pipelines. Feature extraction methods have previously been used to help distinguishing between good and bad quality spectra to apply subsequent supervised pattern recognition techniques. In this study, we apply feature extraction differently and use a variant of a method for blind source separation, namely Convex Non-Negative Matrix Factorization, to unveil MRS signal sources in a completely unsupervised way. We hypothesize that, while most sources will correspond to the different tumor patterns, some of them will reflect signal artefacts. The experimental work reported in this paper, analyzing a combined short and long echo time 1 H-MRS database of more than 2000 spectra acquired at 1.5T and corresponding to different tumor types and other anomalous masses, provides a first proof of concept that points to the possible validity of this approach.

Modularized convex nonnegative matrix factorization for community detection in signed and unsigned networks

NMF-based models in unsigned networks, the links of which are positive links only, have been applied in many aspects, such as community detection, link prediction, etc. However, NMF has been under-explored for community discovery in signed networks due to its constraint of non-negativity. Also, there are few related studies which could find out accurate partitions on both signed and unsigned networks due to their difference of community structure. In this paper, we propose a novel modularized convex nonnegative matrix factorization model which combines signed modularized information with convex NMF model, improving the accuracy of community detection in signed and unsigned networks. As for model selection, we extend the modularity density to signed networks and employ the signed modularity density to determine the number of communities automatically. Finally, the effectiveness of our model is verified on both synthetic and real-world networks.

Similarity Constrained Convex Nonnegative Matrix Factorization for Hyperspectral Anomaly Detection

Hyperspectral anomaly detection is very important in the remote sensing domain. The representation-based anomaly method is one of the most important hyperspectral anomaly detection methods, which uses reconstruction errors (REs) to detect anomalies. REs are affected by the basis matrix and its corresponding coefficient matrix. Mixed pixels exist because of the low-spatial resolution of hyperspectral images. The RE is not large enough to correctly distinguish the pixel difficult to classify when the basis matrix is composed of pixels. Moreover, its corresponding coefficients cannot indicate whether pixels are pure or mixed and the abundances of mixed pixels. To address the above-mentioned problems, endmembers referring to pure or relatively pure spectral signatures are explored to build the basis matrix. The RE based on the basis matrix of endmembers is much larger for the anomalous pixel difficult to correctly classify. Furthermore, its corresponding coefficient matrix of endmembers has physical meanings. Hence, a novel hyperspectral anomaly detection based on similarity constrained convex nonnegative matrix factorization is proposed from the perspective of endmembers for the first time. First, convex nonnegative matrix factorization (CNMF) is employed to obtain endmembers of background. Then, CNMF is constrained by the similarity regularization that considers different contributions of endmembers to the pixel under test to acquire the more accurate and meaningful coefficient matrix. Finally, anomalies are detected by calculating REs. The proposed algorithm is verified on both simulated and real data sets. Experimental results show that our proposed algorithm outperforms other state-of-the-art algorithms.

Research of SWNMF with New Iteration Rules for Facial Feature Extraction and Recognition

Weighted nonnegative matrix factorization (WNMF) is a technology for feature extraction, which can extract the feature of face dataset, and then the feature can be recognized by the classifier. To improve the performance of WNMF for feature extraction, a new iteration rule is proposed in this paper. Meanwhile, the base matrix U is sparse based on the threshold, and the new method is named sparse weighted nonnegative matrix factorization (SWNMF). The new iteration rules are based on the smaller iteration steps, thus, the search is more precise, therefore, the recognition rate can be improved. In addition, the sparse method based on the threshold is adopted to update the base matrix U, which can make the extracted feature more sparse and concentrate, and then easier to recognize. The SWNMF method is applied on the ORL and JAFEE datasets, and from the experiment results we can find that the recognition rates are improved extensively based on the new iteration rules proposed in this paper. The recognition rate of new SWNMF method reached 98% for ORL face database and 100% for JAFEE face database, respectively, which are higher than the PCA method, the sparse nonnegative matrix factorization (SNMF) method, the convex non-negative matrix factorization (CNMF) method and multi-layer NMF method.

Low-rank network signatures in the triple network separate schizophrenia and major depressive disorder.

Brain imaging studies have revealed that functional and structural brain connectivity in the so-called triple network (i.e., default mode network (DMN), salience network (SN) and central executive network (CEN)) are consistently altered in schizophrenia. However, similar changes have also been found in patients with major depressive disorder, prompting the question of specific triple network signatures for the two disorders. In this study, we proposed Supervised Convex Nonnegative Matrix Factorization (SCNMF) to extract distributed multi-modal brain patterns. These patterns distinguish schizophrenia and major depressive disorder in a latent low-dimensional space of the triple brain network. Specifically, 21 patients of schizophrenia and 25 patients of major depressive disorder were assessed by T1-weighted, diffusion-weighted, and resting-state functional MRIs. Individual structural and functional connectivity networks, based on pre-defined regions of the triple network were constructed, respectively. Afterwards, SCNMF was employed to extract the discriminative patterns. Experiments indicate that SCNMF allows extracting the low-rank discriminative patterns between the two disorders, achieving a classification accuracy of 82.6% based on the extracted functional and structural abnormalities with support vector machine. Experimental results show the specific brain patterns for schizophrenia and major depressive disorder that are multi-modal, complex, and distributed in the triple network. Parts of the prefrontal cortex including superior frontal gyri showed variation between patients with schizophrenia and major depression due to structural properties. In terms of functional properties, the middle cingulate cortex, inferior parietal lobule, and cingulate cortex were the most discriminative regions.

Subspace clustering guided convex nonnegative matrix factorization

Abstract As one of the most important information of the data, the geometry structure information is usually modeled by a similarity graph to enforce the effectiveness of nonnegative matrix factorization (NMF). However, pairwise distance based graph is sensitive to noise and can not capture the subspace structure of the data. Reconstruction coefficients based graph can capture the subspace structure of the data, but the procedure of building the representation based graph is usually independent to the framework of NMF. To address this issue, a novel subspace clustering guided convex nonnegative matrix factorization (SC-CNMF) is proposed. In this NMF framework, the nonnegative subspace clustering is incorporated to learning the representation based graph, and meanwhile, a convex nonnegative matrix factorization is also updated simultaneously. To tackle the noise influence of the dataset, only k largest entries of each representation are kept in the subspace clustering. To capture the complicated geometry structure of the data, multiple centroids are also introduced to describe each cluster. Additionally, a row constraint is used to remove the relevance among the rows of the encoding matrix, which can help to improve the clustering performance of the proposed model. For the proposed NMF framework, two different objective functions with different optimizing schemes are designed. Image clustering experiments are conducted to demonstrate the effectiveness of the proposed methods on several datasets and compared with some related works based on NMF together with k-means clustering method and PCA as baseline.