Nonsmooth Nonnegative Matrix Factorization Research Articles

Interpretable machine learning model using Extreme Gradient Boosting with Nonnegative Matrix Factorization improves the accuracy of arrhythmic risk prediction in Brugada Syndrome

Abstract Background Both depolarization and repolarization abnormalities contribute to ventricular arrhythmogenesis in Brugada syndrome (BrS). In this study, we tested the hypothesis that incorporating latent features extracted by various nonnegative matrix factorization (NML) techniques into an interpretable machine learning (IML) prediction model can outperform IML models without latent variables and logistic regression model. Methods This study was based on a published anonymised dataset of BrS patients from the Hong Kong, China. XGBoost was selected as the IML model, with and without incorporating latent features using 11 different NMF techniques: Bayesian nonnegative matrix factorization (BNMF), Iterated Conditional Modes nonnegative matrix factorization (ICM), Fisher Nonnegative Matrix Factorization for learning Local features (LFNMF), Alternating Nonnegative Least Squares Matrix Factorization Using Projected Gradient (bound constrained optimization) method for each subproblem (LSNMF), Non-smooth Nonnegative Matrix Factorization (NSNMF), Probabilistic Nonnegative Matrix Factorization (PMF), Probabilistic Sparse Matrix Factorization (PSMF), Sparse Nonnegative Matrix Factorization (SNMF) based on alternating nonnegativity constrained least squares, Sparse Network-Regularized Multiple Nonnegative Matrix Factorization (SNMNMF), Penalized Matrix Factorization for Constrained Clustering (PMFCC) and Separable Nonnegative Matrix Factorization (SepNMF). Results A total of 548 patients were included (7.3% females, age at diagnosis: 51.0 [38.0-61.0] years old. Of these, 66 suffered from spontaneous ventricular tachyarrhythmias over 84±55 months follow-up. The baseline model using multivariable logistic regression achieved an area under the curve (AUC) of 0.78 [0.72-0.85], which was improved to 0.88 [0.83-0.93] for the IML model without NMF. The AUC was further increased by incorporating additional latent variables extracted using NSNMF (0.95 [0.92-0.98]) and PMFCC (0.94 [0.88-1.00]). Conclusion Incorporation of latent variables by different NMF techniques into an IML prediction model significantly improved the accuracy of risk prediction in BrS.

Disease patterns of coronary heart disease and type 2 diabetes harbored distinct and shared genetic architecture

Coronary heart disease (CHD) and type 2 diabetes (T2D) are two complex diseases with complex interrelationships. However, the genetic architecture of the two diseases is often studied independently by the individual single-nucleotide polymorphism (SNP) approach. Here, we presented a genotypic-phenotypic framework for deciphering the genetic architecture underlying the disease patterns of CHD and T2D. A data-driven SNP-set approach was performed in a genome-wide association study consisting of subpopulations with different disease patterns of CHD and T2D (comorbidity, CHD without T2D, T2D without CHD and all none). We applied nonsmooth nonnegative matrix factorization (nsNMF) clustering to generate SNP sets interacting the information of SNP and subject. Relationships between SNP sets and phenotype sets harboring different disease patterns were then assessed, and we further co-clustered the SNP sets into a genetic network to topologically elucidate the genetic architecture composed of SNP sets. We identified 23 non-identical SNP sets with significant association with CHD or T2D (SNP-set based association test, P < 3.70 × [Formula: see text]). Among them, disease patterns involving CHD and T2D were related to distinct SNP sets (Hypergeometric test, P < 2.17 × [Formula: see text]). Accordingly, numerous genes (e.g., KLKs, GRM8, SHANK2) and pathways (e.g., fatty acid metabolism) were diversely implicated in different subtypes and related pathophysiological processes. Finally, we showed that the genetic architecture for disease patterns of CHD and T2D was composed of disjoint genetic networks (heterogeneity), with common genes contributing to it (pleiotropy). The SNP-set approach deciphered the complexity of both genotype and phenotype as well as their complex relationships. Different disease patterns of CHD and T2D share distinct genetic architectures, for which lipid metabolism related to fibrosis may be an atherogenic pathway that is specifically activated by diabetes. Our findings provide new insights for exploring new biological pathways.

DNA Methylation Profiles of Ovarian Clear Cell Carcinoma.

Ovarian clear cell carcinoma (OCCC) is a rare ovarian cancer histotype that tends to be resistant to standard platinum-based chemotherapeutics. We sought to better understand the role of DNA methylation in clinical and biological subclassification of OCCC. We interrogated genome-wide methylation using DNA from fresh frozen tumors from 271 cases, applied nonsmooth nonnegative matrix factorization (nsNMF) clustering, and evaluated clinical associations and biological pathways. Two approximately equally sized clusters that associated with several clinical features were identified. Compared with Cluster 2 (N = 137), Cluster 1 cases (N = 134) presented at a more advanced stage, were less likely to be of Asian ancestry, and tended to have poorer outcomes including macroscopic residual disease following primary debulking surgery (P < 0.10). Subset analyses of targeted tumor sequencing and IHC data revealed that Cluster 1 tumors showed TP53 mutation and abnormal p53 expression, and Cluster 2 tumors showed aneuploidy and ARID1A/PIK3CA mutation (P < 0.05). Cluster-defining CpGs included 1,388 CpGs residing within 200 bp of the transcription start sites of 977 genes; 38% of these genes (N = 369 genes) were differentially expressed across cluster in transcriptomic subset analysis (P < 10-4). Differentially expressed genes were enriched for six immune-related pathways, including IFNα and IFNγ responses (P < 10-6). DNA methylation clusters in OCCC correlate with disease features and gene expression patterns among immune pathways. This work serves as a foundation for integrative analyses that better understand the complex biology of OCCC in an effort to improve potential for development of targeted therapeutics.

非平滑非负矩阵分解解析土壤多环芳烃三维荧光光谱

非平滑非负矩阵分解解析土壤多环芳烃三维荧光光谱

Deep nonsmooth nonnegative matrix factorization network with semi-supervised learning for SAR image change detection

In the paper, we propose a deep nonsmooth nonnegative matrix factorization (nsNMF) network with semi-supervised learning for synthetic aperture radar (SAR) image change detection. In most of the existing deep-NMF-based models, the nonnegative matrix is linearly decomposed layer by layer, which may fail to characterize the nonlinearities in complex data. As such, a nonlinear deep nsNMF model is first built for learning hierarchical, nonlinear, and localized data representations. Meanwhile, in view of its good generalization performance and low computational complexity, extreme learning machine (ELM) is integrated into the nonlinear deep nsNMF model to construct a deep nsNMF network for satisfactory classification. More importantly, since it is difficult to acquire more labeled samples in practice, semi-supervised learning strategy is proposed to make use of partially labeled data for training. The learning process of the proposed network consists of pretraining stage and fine-tuning stage, in which the former pretrains all decomposed matrices layer by layer and the latter aims to reduce the total reconstruction error by using the mini-batch gradient descent algorithm. The experimental results on four pairs of SAR images demonstrate the effectiveness of the proposed method.

Cancer classification and pathway discovery using non-negative matrix factorization.

Extracting genetic information from a full range of sequencing data is important for understanding disease. We propose a novel method to effectively explore the landscape of genetic mutations and aggregate them to predict cancer type. We applied non-smooth non-negative matrix factorization (nsNMF) and support vector machine (SVM) to utilize the full range of sequencing data, aiming to better aggregate genetic mutations and improve their power to predict disease type. More specifically, we introduce a novel classifier to distinguish cancer types using somatic mutations obtained from whole-exome sequencing data. Mutations were identified from multiple cancers and scored using SIFT, PP2, and CADD, and collapsed at the individual gene level. nsNMF was then applied to reduce dimensionality and obtain coefficient and basis matrices. A feature matrix was derived from the obtained matrices to train a classifier for cancer type classification with the SVM model. We have demonstrated that the classifier was able to distinguish four cancer types with reasonable accuracy. In five-fold cross-validations using mutation counts as features, the average prediction accuracy was 80% (SEM = 0.1%), significantly outperforming baselines and outperforming models using mutation scores as features. Using the factor matrices derived from the nsNMF, we identified multiple genes and pathways that are significantly associated with each cancer type. This study presents a generic and complete pipeline to study the associations between somatic mutations and cancers. The proposed method can be adapted to other studies for disease status classification and pathway discovery.

Abstract P3-06-06: Novel breast cancer mutational signatures

Abstract Background: Our understanding of the biological processes that generate somatic mutations in breast cancer has increased markedly over the past five years. Using the catalog of somatic mutations present in cancer genomes, over 30 “mutational signatures” have been produced. While these provide important insights into the processes responsible for somatic mutation, gaps remain, and the etiology of several signatures remains unknown. Methods: We have developed a new method in which the specific nucleotide change (e.g., C&amp;gt;T), the codon that each mutation falls in (e.g., GCT), the position in the codon (e.g., 2), and the nucleotides immediately 5' and 3' of the mutation (e.g., 5': C; 3': G) are all considered. The summary of these mutation characteristics forms a mutational profile for each tissue sample. Putting multiple samples' profiles together forms a sparse matrix with the number of samples as rows and the mutation characteristics as columns. Nonsmooth nonnegative matrix factorization was then applied to enable the discovery of intrinsic patterns in this sparse matrix. Results: Using somatic mutations identified in 1017 breast cancer tissues from The Cancer Genome Atlas (TCGA), we have identified four mutational signatures. Signature A correlates with the well-defined APOBEC signatures and signature B with the “aging” signature, which is the result 5-methylcytosine hydrolysis. Signature C and signature D are potentially new signatures. Signature D is enriched with C:G&amp;gt;A:T mutations; these mostly occur in the middle position of codons, and are enriched with GG(CC) either 5' or 3' of the mutation's sequence context. G&amp;gt;T mutations are known to occur as a consequence of oxidative damage that is not repaired. Guanines are vulnerable as they have the highest vertical oxidation potential of the nucleobases. The 5' guanines in GG sites are especially reactive. We hypothesize that Signature D results from oxidative mutagenesis. When correlated with clinical phenotypes, the basal subtype is clearly enriched for tumors with the Signature D mutation pattern (exposure level is in 169 basal tumors and in 797 non-basal tumors, p=&amp;lt;0.01), suggesting an etiologic link with basal-like breast cancer. G&amp;gt;T somatic mutations in breast cancer mainly take place during cell replication rather than during transcription. In the normal breast, epithelial cell replication occurs during the luteal phase of the menstrual cycle and during pregnancy, primarily under the direction of progesterone (P). P binds to its receptor (PR) in a subpopulation of PR positive cells where it initiates the transcription of genes including RANKL, with resultant paracrine stimulation (through RANK), of the NF-κB signaling pathway in neighboring cells. The four RANKL genes (TOP2A, MKI67, PBK, CDK1), defined by Nolan et al., are all positively associated with the signature D (P-values &amp;lt; 0.05), suggesting that this type of mutagenesis is associated with RANKL pathway upregulation. Conclusions: We have identified a potentially new somatic mutational signature, which we have designated as Signature D, which appears to result from exposure of DNA to oxidative stress during replication. It is associated with the basal subtype of breast cancer as well as RANKL- NF-κB pathway upregulation. Citation Format: Zeng Z, Vo AH, Luo Y, Khan SA, Clare SE. Novel breast cancer mutational signatures [abstract]. In: Proceedings of the 2018 San Antonio Breast Cancer Symposium; 2018 Dec 4-8; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2019;79(4 Suppl):Abstract nr P3-06-06.

Iterative Weighted Non-smooth Non-negative Matrix Factorization for Face Recognition

Non-negative Matrix Factorization (NMF) is a part-based image representation method. It comes from the intuitive idea that entire face image can be constructed by combining several parts. In this paper, we propose a framework for face recognition by finding localized, part-based representations, denoted “Iterative weighted non-smooth non-negative matrix factorization” (IWNS-NMF). A new cost function is proposed in order to incorporate sparsity which is controlled by a specific parameter and weights of feature coefficients. This method extracts highly localized patterns, which generally improves the capability of face recognition. After extracting patterns by IWNS-NMF, we use principle component analysis to reduce dimension for classification by linear SVM. The Recognition rates on ORL, YALE and JAFFE datasets were 97.5, 93.33 and 87.8%, respectively. Comparisons to the related methods in the literature indicate that the proposed IWNS-NMF method achieves higher face recognition performance than NMF, NS-NMF, Local NMF and SNMF.

Learning the Hierarchical Parts of Objects by Deep Non-Smooth Nonnegative Matrix Factorization

Nonsmooth Nonnegative Matrix Factorization (nsNMF) is capable of producing more localized, less overlapped feature representations than other variants of NMF while keeping satisfactory fit to data. However, nsNMF as well as other existing NMF methods is incompetent to learn hierarchical features of complex data due to its shallow structure. To fill this gap, we propose a deep nsNMF method coined by the fact that it possesses a deeper architecture compared with standard nsNMF. The deep nsNMF not only gives parts-based features due to the nonnegativity constraints, but also creates higher-level, more abstract features by combing lower-level ones. The in-depth description of how deep architecture can help to efficiently discover abstract features in dnsNMF is presented. And we also show that the deep nsNMF has close relationship with the deep autoencoder, suggesting that the proposed model inherits the major advantages from both deep learning and NMF. Extensive experiments demonstrate the standout performance of the proposed method in clustering analysis.

Adaptive Method for Nonsmooth Nonnegative Matrix Factorization.

Nonnegative matrix factorization (NMF) is an emerging tool for meaningful low-rank matrix representation. In NMF, explicit constraints are usually required, such that NMF generates desired products (or factorizations), especially when the products have significant sparseness features. It is known that the ability of NMF in learning sparse representation can be improved by embedding a smoothness factor between the products. Motivated by this result, we propose an adaptive nonsmooth NMF (Ans-NMF) method in this paper. In our method, the embedded factor is obtained by using a data-related approach, so it matches well with the underlying products, implying a superior faithfulness of the representations. Besides, due to the usage of an adaptive selection scheme to this factor, the sparseness of the products can be separately constrained, leading to wider applicability and interpretability. Furthermore, since the adaptive selection scheme is processed through solving a series of typical linear programming problems, it can be easily implemented. Simulations using computer-generated data and real-world data show the advantages of the proposed Ans-NMF method over the state-of-the-art methods.

A Fast Non-Smooth Nonnegative Matrix Factorization for Learning Sparse Representation

Nonnegative matrix factorization (NMF) is a hot topic in machine learning and data processing. Recently, a constrained version, non-smooth NMF (NsNMF), shows a great potential in learning meaningful sparse representation of the observed data. However, it suffers from a slow linear convergence rate, discouraging its applications to large-scale data representation. In this paper, a fast NsNMF (FNsNMF) algorithm is proposed to speed up NsNMF. In the proposed method, it first shows that the cost function of the derived sub-problem is convex and the corresponding gradient is Lipschitz continuous. Then, the optimization to this function is replaced by solving a proximal function, which is designed based on the Lipschitz constant and can be solved through utilizing a constructed fast convergent sequence. Due to the usage of the proximal function and its efficient optimization, our method can achieve a nonlinear convergence rate, much faster than NsNMF. Simulations in both computer generated data and the real-world data show the advantages of our algorithm over the compared methods.

Exploring matrix factorization techniques for significant genes identification of Alzheimer’s disease microarray gene expression data

BackgroundThe wide use of high-throughput DNA microarray technology provide an increasingly detailed view of human transcriptome from hundreds to thousands of genes. Although biomedical researchers typically design microarray experiments to explore specific biological contexts, the relationships between genes are hard to identified because they are complex and noisy high-dimensional data and are often hindered by low statistical power. The main challenge now is to extract valuable biological information from the colossal amount of data to gain insight into biological processes and the mechanisms of human disease. To overcome the challenge requires mathematical and computational methods that are versatile enough to capture the underlying biological features and simple enough to be applied efficiently to large datasets.MethodsUnsupervised machine learning approaches provide new and efficient analysis of gene expression profiles. In our study, two unsupervised knowledge-based matrix factorization methods, independent component analysis (ICA) and nonnegative matrix factorization (NMF) are integrated to identify significant genes and related pathways in microarray gene expression dataset of Alzheimer’s disease. The advantage of these two approaches is they can be performed as a biclustering method by which genes and conditions can be clustered simultaneously. Furthermore, they can group genes into different categories for identifying related diagnostic pathways and regulatory networks. The difference between these two method lies in ICA assume statistical independence of the expression modes, while NMF need positivity constrains to generate localized gene expression profiles.ResultsIn our work, we performed FastICA and non-smooth NMF methods on DNA microarray gene expression data of Alzheimer’s disease respectively. The simulation results shows that both of the methods can clearly classify severe AD samples from control samples, and the biological analysis of the identified significant genes and their related pathways demonstrated that these genes play a prominent role in AD and relate the activation patterns to AD phenotypes. It is validated that the combination of these two methods is efficient.ConclusionsUnsupervised matrix factorization methods provide efficient tools to analyze high-throughput microarray dataset. According to the facts that different unsupervised approaches explore correlations in the high-dimensional data space and identify relevant subspace base on different hypotheses, integrating these methods to explore the underlying biological information from microarray dataset is an efficient approach. By combining the significant genes identified by both ICA and NMF, the biological analysis shows great efficient for elucidating the molecular taxonomy of Alzheimer’s disease and enable better experimental design to further identify potential pathways and therapeutic targets of AD.

A comparative study of facial appearance modeling methods for active appearance models

Active appearance models (AAMs) have been widely used in many face modeling and facial feature extraction methods. One of the problems of AAMs is that it is difficult to model a sufficiently wide range of human facial appearances, the pattern of intensities across a face image patch. Previous researches have used principal component analysis (PCA) for facial appearance modeling, but there has been little analysis and comparison between PCA and many other facial appearance modeling methods such as non-negative matrix factorization (NMF), local NMF (LNMF), and non-smooth NMF (ns-NMF). The main contribution of this paper is to find a suitable facial appearance modeling method for AAMs by a comparative study. In the experiments, PCA, NMF, LNMF, and ns-NMF were used to produce the appearance model of the AAMs and the root mean square (RMS) errors of the detected feature points were analyzed using the AR and BERC face databases. Experimental results showed that (1) if the appearance variations of testing face images were relatively non-sparser than those of training face images, the non-sparse methods (PCA, NMF) based AAMs outperformed the sparse methods (nsNMF, LNMF) based AAMs. (2) If the appearance variations of testing face images are relatively sparser than those of training face images, the sparse methods (nsNMF) based AAMs outperformed the non-sparse methods (PCA, NMF) based AAMs.

Constrained Nonnegative Matrix Factorization for Hyperspectral Unmixing

Hyperspectral unmixing is a process to identify the constituent materials and estimate the corresponding fractions from the mixture. During the last few years, nonnegative matrix factorization (NMF), as a suitable candidate for the linear spectral mixture model, has been applied to unmix hyperspectral data. Unfortunately, the local minima caused by the nonconvexity of the objective function makes the solution nonunique, thus only the nonnegativity constraint is not sufficient enough to lead to a well-defined problem. Therefore, in this paper, two inherent characteristics of hyperspectral data, piecewise smoothness (both temporal and spatial) of spectral data and sparseness of abundance fraction of every material, are introduced to NMF. The adaptive potential function from discontinuity adaptive Markov random field model is used to describe the smoothness constraint while preserving discontinuities in spectral data. At the same time, two NMF algorithms, nonsmooth NMF and NMF with sparseness constraint, are used to quantify the degree of sparseness of material abundances. A gradient-based optimization algorithm is presented, and the monotonic convergence of the algorithm is proved. Three important facts are exploited in our method: First, both the spectra and abundances are nonnegative; second, the variation of the material spectra and abundance images is piecewise smooth in wavelength and spatial spaces, respectively; third, the abundance distribution of each material is almost sparse in the scene. Experiments using synthetic and real data demonstrate that the proposed algorithm provides an effective unsupervised technique for hyperspectral unmixing.

Nonsmooth nonnegative matrix factorization (nsNMF)

We propose a novel nonnegative matrix factorization model that aims at finding localized, part-based, representations of nonnegative multivariate data items. Unlike the classical nonnegative matrix factorization (NMF) technique, this new model, denoted "nonsmooth nonnegative matrix factorization" (nsNMF), corresponds to the optimization of an unambiguous cost function designed to explicitly represent sparseness, in the form of nonsmoothness, which is controlled by a single parameter. In general, this method produces a set of basis and encoding vectors that are not only capable of representing the original data, but they also extract highly localized patterns, which generally lend themselves to improved interpretability. The properties of this new method are illustrated with several data sets. Comparisons to previously published methods show that the new nsNMF method has some advantages in keeping faithfulness to the data in the achieving a high degree of sparseness for both the estimated basis and the encoding vectors and in better interpretability of the factors.