NMF Algorithms Research Articles

Deep Nonnegative Matrix Factorization With Beta Divergences.

Deep nonnegative matrix factorization (deep NMF) has recently emerged as a valuable technique for extracting multiple layers of features across different scales. However, all existing deep NMF models and algorithms have primarily centered their evaluation on the least squares error, which may not be the most appropriate metric for assessing the quality of approximations on diverse data sets. For instance, when dealing with data types such as audio signals and documents, it is widely acknowledged that ß-divergences offer a more suitable alternative. In this article, we develop new models and algorithms for deep NMF using some ß-divergences, with a focus on the Kullback-Leibler divergence. Subsequently, we apply these techniques to the extraction of facial features, the identification of topics within document collections, and the identification of materials within hyperspectral images.

Applying Machine Learning in Marketing: An Analysis Using the NMF and k-Means Algorithms

The integration of machine learning (ML) techniques into marketing strategies has become increasingly relevant in modern business. Utilizing scientific manuscripts indexed in the Scopus database, this article explores how this integration is being carried out. Initially, a focused search is undertaken for academic articles containing both the terms “machine learning” and “marketing” in their titles, which yields a pool of papers. These papers have been processed using the Supabase platform. The process has included steps like text refinement and feature extraction. In addition, our study uses two key ML methodologies: topic modeling through NMF and a comparative analysis utilizing the k-means clustering algorithm. Through this analysis, three distinct clusters emerged, thus clarifying how ML techniques are influencing marketing strategies, from enhancing customer segmentation practices to optimizing the effectiveness of advertising campaigns.

Increasing the Effectiveness of Prediction in Recommendation Engines Based on Collaborative Filtering

In the era of information abundance, the demand for personalized content recommendations has become paramount. Recommendation engines, particularly those employing collaborative filtering, play a pivotal role in delivering tailored suggestions based on user preferences. As technology evolves, the need to enhance the effectiveness of prediction algorithms within these engines becomes increasingly crucial. This research endeavors to contribute to this evolving landscape by delving into collaborative filtering methodologies, identifying challenges, and proposing novel strategies to elevate the accuracy and relevance of predictions in recommendation systems. Through this exploration, we aim to not only refine existing models but also pave the way for more sophisticated and reliable personalized content recommendations.
 This research aims to enhance prediction accuracy in recommendation engines utilizing collaborative filtering. Through an in-depth exploration of collaborative filtering techniques, we propose innovative approaches to improve the effectiveness of predictions. Our study addresses key challenges in collaborative filtering models, offering insights into refined algorithms and methodologies. By fine-tuning the collaborative filtering process, we anticipate a substantial boost in the overall performance of recommendation engines, ultimately advancing the field of personalized content suggestion. The simulation is performed using Java language and using two datasets Movie Lens 1M and Movie Lens 100K.The proposed model was evaluated using the Mean Absolute Error, Precision, and Recall.
 The proposed model achieved a mean absolute error value ranging between 0.78 and 0.84 using the Movie Lens 100K dataset, and a mean absolute error value ranging between 0.72 and 0.74 using the Movie Lens 1M dataset for different values of the number of user groups. As for precision and recall, the precision of the proposed model ranged between 0.97 and 0.985 using the Movie Lens 100K data set, and a precision value ranging between 0.944 and 0.954 using the Movie Lens 1M data set, also for different values of the number of user groups.
 As for the recall results, the proposed model achieved a recall value ranging between 0.755 and 0.85 using the Movie Lens 100K dataset, and a recall value ranging between 0.72 and 0.75 using the Movie Lens 100K dataset, also for different values of the number of user groups. These results were compared with the PMF, HPF, and NMF algorithms, where the proposed model proved its clear superiority over these algorithms. Using this analysis of the matrix allows us to obtain a good prediction accuracy of users' preferences and to find common groups of people with similar preferences.

Necroptosis-related KLRB1 was a potent tumor suppressor and immunotherapy determinant in breast cancer

Breast cancer is a multifaceted and diverse illness that impacts millions of people globally. Identifying the underlying causes of BRCA and creating efficient treatment plans are urgent. Necroptosis is widely involved in cancer development. However, the specific roles of necroptosis in cancer immunotherapy of breast cancer have not been explored. In this study, we aim to establish the connection between necroptosis and immunotherapy in BRCA. TCGA, METABRIC, GSE103091, GSE159956, and GSE96058 were included for bioinformatics analysis. NMF and CoxBoost algorithms were used to develop the necroptosis-related patterns and model, respectively. A necroptosis-related model was developed and determined KLRB1 as a critical tumor suppressor by in vitro validation. The mutation characteristics, immune characteristics, and molecular functions of KLRB1 were explored. We further examined how necroptosis-related KLRB1 functions in BRCA as a powerful tumor suppressor and regulates the activity of macrophages by in vitro validation, including CCK8, EdU, and Transwell assays. KLRB1 was also revealed to be an immunotherapy determinant.

Constrained Nonnegative Matrix Factorization Based on Label Propagation for Data Representation

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Nonnegative matrix factorization</i> (NMF) algorithms are a series of dimensional reduction techniques widely used in data preprocessing. To improve the performance of clustering and the discrimination of the low-dimensional representation in NMF, we proposed a novel semi-supervised <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">constrained nonnegative matrix factorization based on label propagation</i> (LpCNMF). Specifically, the proposed LpCNMF adopts graph and label propagation as regularization terms, then makes use of a small amount of labeled data to predict the label information of the unlabeled data and finally obtains a predictive membership matrix with more label information. At the same time, we introduce an efficient alternating iterative algorithm to solve the optimization problem of the objective function in the LpCNMF. Unlike other NMF algorithms that only update the basis and coefficient matrices, the LpCNMF algorithm increases the update of the predictive membership matrix obtained by label propagation. Experimental results on various benchmark datasets demonstrate the superiority of our algorithm over existing state-of-the-art NMF algorithms. The code is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ycwang-libra/LpCNMF_github.git.</uri>

Abstract 2289: Comprehensive multiomic analysis reveals different subtypes in neuroendocrine tumors from multiple origins

Abstract Neuroendocrine neoplasms (NENs) are a heterogeneous group of tumors with a wide anatomical distribution and different clinical behaviors that share common “neuro” and “endocrine” properties. They are classified in well differentiated G1, G2 and G3 neuroendocrine tumors (NETs), or poorly differentiated neuroendocrine carcinomas (NECs) which are all G3. Some studies have assessed the NET molecular landscape of certain primary sites, but no cross-sites molecular assessment has been performed to date. Thus, we aimed to perform a comprehensive transcriptomic and methylomic characterization of large series of NENs of different anatomical origins in order to identify molecular subgroups, unravel genes and/or pathways involved, and explore their potential clinical relevance. Transcriptomic and methylation data were obtained from tumor samples of 181 and 99 patients with NENs (37.02%G1; 54.7%G2 and 8.29%G3) using Clariom S Human microarrays and EPIC methylation arrays, respectively. Most common primary tumor site was the lung (43.1%), followed by the small intestine (35.4%), large intestine (8.8%), stomach (4.4%), pancreas (1.7%), other localizations (3.9%), and unknown primary (2.8%). Tumors were clustered using transcriptomic and methylation data by NMF and hierarchical clustering algorithms respectively, and differential expression (limma analysis) and gene set enrichment analysis (GSEA) was applied to transcriptomic data clusters NENs were classified into three different clusters. Two of them (C1 and C3) showed an inverse transcriptomic profile, while C2 presented a mixed profile. We identified that C3 had the highest expression levels of neuroendocrine-related genes (somatostatin receptor SSTR5 LFC: 0.77 ; FDR: 1.52x10e-25 ; serotonin receptor HTR1A LFC: 0.79 ; FDR: 1.20X10e-20 ; VIP LFC: 0.64 ; FDR= 1.41x10e-10 etc), in lung primary tumor samples (50%) and methylation cluster 1 (82.14%). On the other hand, C1 was enriched in some important cancer-related pathways ( MYC targets NES -6.37 FDR &amp;lt; 10-3; MTORC1 NES -3.78 FDR &amp;lt; 10-3) and oxidative phosphorylation (NES -4.87 FDR &amp;lt; 10-3) ; and showed a higher proportion of small intestine primary tumors (46.34%), Grade 2 tumors (73.17%) and methylation cluster 2 (75%). NENs of different primary sites can be stratified into two major transcriptomic clusters, with a third cluster having a mixed expression profile. The groups with extreme transcriptomic signatures were enriched in neuroendocrine-related genes or principal cancer pathways. These results suggest a dedifferentiation from a more indolent neuroendocrine phenotype to more aggressive proliferative tumor types that could be driven partially with the methylation profile. Citation Format: Carlos Carretero Puche, Beatriz Soldevilla, Beatriz Gil, Anna La Salvia, Alberto Lens-Pardo, Paula Espinosa, Beatriz Anton-Pascual, Gonzalo Gómez-López, Fátima Al-Shahrour, Rocío García-Carbonero. Comprehensive multiomic analysis reveals different subtypes in neuroendocrine tumors from multiple origins [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 2289.

A survey of deep nonnegative matrix factorization

Deep Nonnegative Matrix Factorization (Deep NMF) is an effective strategy for feature extraction in recent years. By decomposing the matrix recurrently on account of the NMF algorithms, we obtain a hierarchical neural network structure as well as exploring more interpretable representations of the data. This paper mainly focuses on some theoretical researches with respect to Deep NMF, where the basic models, optimization methods, properties together with its extensions and generalizations are included systematically. We partition the Deep NMF algorithms into five categories: Deep NMF, Constrained Deep NMF, Generalized Deep NMF, Multi-View Deep Matrix Factorization (MF), the association between deep neural network (DNN) and NMF. Besides, we investigate the clustering performance of Deep NMF Algorithms on some face databases. Then the design principles, major steps, relationships, application domains and evolution of Deep NMF methods are comprehensively analyzed. Moreover, some open problems of Deep NMF are discussed.

Assessment of nonnegative matrix factorization algorithms for electroencephalography spectral analysis

BackgroundNonnegative matrix factorization (NMF) has been successfully used for electroencephalography (EEG) spectral analysis. Since NMF was proposed in the 1990s, many adaptive algorithms have been developed. However, the performance of their use in EEG data analysis has not been fully compared. Here, we provide a comparison of four NMF algorithms in terms of accuracy of estimation, stability (repeatability of the results) and time complexity of algorithms with simulated data. In the practical application of NMF algorithms, stability plays an important role, which was an emphasis in the comparison. A Hierarchical clustering algorithm was implemented to evaluate the stability of NMF algorithms.ResultsIn simulation-based comprehensive analysis of fit, stability, accuracy of estimation and time complexity, hierarchical alternating least squares (HALS) low-rank NMF algorithm (lraNMF_HALS) outperformed the other three NMF algorithms. In the application of lraNMF_HALS for real resting-state EEG data analysis, stable and interpretable features were extracted.ConclusionBased on the results of assessment, our recommendation is to use lraNMF_HALS, providing the most accurate and robust estimation.

Generative adversarial networks for non-negative matrix factorization in temporal psycho-visual modulation

The image factorization problem is the key challenge in Temporal Psycho-Visual Modulation (TPVM). In this paper, we present an end-to-end learned model for image-based non-negative matrix factorization. We decompose a set of images into a small number of image bases which can be used to reconstruct all the images by linearly combining the bases. During the process, the image bases, as well as their weights for the linear combination, are unknown. The method is based on conditional GAN and a variational sample disturber. Traditional NMF methods suffer from slow computational speed and poor generalization ability. A deep neural network shows potential in these two aspects. We conduct adversarial training in our model to generate better bases for image restoration. Our method outperforms other CNN based methods on several public datasets. Compared to traditional NMF algorithms, our model generates image bases that preserve details. Our model has advantages in speed as well as generalization ability.

Attention-Based Multi-NMF Deep Neural Network with Multimodality Data for Breast Cancer Prognosis Model.

Today, it has become a hot issue in cancer research to make precise prognostic prediction for breast cancer patients, which can not only effectively avoid overtreatment and medical resources waste, but also provide scientific basis to help medical staff and patients family members to make right medical decisions. As well known, cancer is a partly inherited disease with various important biological markers, especially the gene expression profile data and clinical data. Therefore, the accuracy of prediction model can be improved by integrating gene expression profile data and clinical data. In this paper, we proposed an end-to-end model, Attention-based Multi-NMF DNN (AMND), which combines clinical data and gene expression data extracted by Multiple Nonnegative Matrix Factorization algorithms (Multi-NMF) for the prognostic prediction of breast cancer. The innovation of this method is highlighted through using clinical data and combining multiple feature selection methods with the help of Attention mechanism. The results of comprehensive performance evaluation show that the proposed model reports better predictive performances than either models only using data of single modality, e.g., gene or clinical, or models based on any single NMF improved methods which only use one of the NMF algorithms to extract features. The performance of our model is competitive or even better than other previously reported models. Meanwhile, AMND can be extended to the survival prediction of other cancer diseases, providing a new strategy for breast cancer prognostic prediction.

Upgrading Sparse NMF algorithm for blind source separation through Adaptive Parameterized Hybrid Kernel based approach

Whenever and wherever the signals or messages are in mixed form, the Blind Source Separation (BSS) methods are utilized for better separation of sound, image or any other form of source signals from this group of observation or mixed signals without knowing the mixing process and/or the nature of source signals. There are multiple algorithms exploited for the separation purpose. At present, despite the fact that BSS by means of Non-Negative Matrix Factorization is accepted as the perfect approach, both of the convergence of solution space and separation quality still exist as main issues under NMF algorithms. Consequently we develop a new algorithm called Adaptive Parameterized Hybrid Kernel based Sparse NMF (APHKSNMF) which is an advanced form among existing NMF algorithms, for blind source separation to optimize the cited drawbacks. Here this proposed NMF method is applied to only audio mix and mixed image signals for separation, whereas it can further be applied to any other forms of signals. In this method, the Weiner filtering is exploited to eliminate superfluous noises and unrelated background information from the mixed audio and image signals. The parameters like combination of the filtering methods, number of observation signals in comparison with number of source signals and blind source separation methods can effectively eliminate the cocktail party obscurities which were previously leading to bad quality of separation. The presented algorithm attains a higher separation quality and convergence speed when compared with the existing methods such as MVCNMF (Miao and Qi 2007) algorithm or NMFJK (Kim and Park 2008) algorithm and is proved as sturdy over different kinds when we reckon audio and image signals. For the combination of 3 sources and 5 observation audio signals and also with images detachedly, APHKSNMF algorithm showed that their SSIM (Structural Similarity Index), UIQI (Universal Image Quality Index) and Separation time values as 0.9876, 0.8475 and 927.35 s respectively when the existing MVCNMF and NMFJK produced inferior results though they were formerly acknowledged as better methods for Blind Separation of whichever forms of signals.

Supervised non-negative matrix factorization methods for MALDI imaging applications.

MotivationNon-negative matrix factorization (NMF) is a common tool for obtaining low-rank approximations of non-negative data matrices and has been widely used in machine learning, e.g. for supporting feature extraction in high-dimensional classification tasks. In its classical form, NMF is an unsupervised method, i.e. the class labels of the training data are not used when computing the NMF. However, incorporating the classification labels into the NMF algorithms allows to specifically guide them toward the extraction of data patterns relevant for discriminating the respective classes. This approach is particularly suited for the analysis of mass spectrometry imaging (MSI) data in clinical applications, such as tumor typing and classification, which are among the most challenging tasks in pathology. Thus, we investigate algorithms for extracting tumor-specific spectral patterns from MSI data by NMF methods.ResultsIn this article, we incorporate a priori class labels into the NMF cost functional by adding appropriate supervised penalty terms. Numerical experiments on a MALDI imaging dataset confirm that the novel supervised NMF methods lead to significantly better classification accuracy and stability as compared with other standard approaches.Availability and implementaton https://gitlab.informatik.uni-bremen.de/digipath/Supervised_NMF_Methods_for_MALDI.git Supplementary information Supplementary data are available at Bioinformatics online.

Development of Reliable and Stable QL1-NMF Algorithm for Analyzing Environmental ELF Magnetic Signals

We previously developed two NMF algorithms (QL1-NMF1 and QL1-NMF2) using the quasi-L1 norm for analyzing environmental ELF magnetic field measurements. When the data included many outliers, the QL1-NMF algorithms returned better results than other BSS algorithms using the L1 norm. However, the derivative of the cost function in QL1-NMF1 was not based on a monotonically increasing function. This problem decreased the validity of the algorithm. QL1-NMF2 had serious problems with stability though it was based on a monotonically increasing derivative. The method therefore required an improvement of validity and stability. In the work described in this paper, we introduced new update functions that were based on a monotonically increasing derivative. Computer simulation results and real data results confirmed the new algorithm worked more stability than the previous one. Moreover, we showed the new algorithm was fast and accurate.

Cross-corpus speech emotion recognition based on transfer non-negative matrix factorization

Automatic emotion recognition from speech has received an increasing amount of interest in recent years, and many speech emotion recognition methods have been presented, in which the training and testing procedures are often conducted on the same corpus. However, in practice, the training and testing speech utterances are collected from different conditions or devices, which will have adverse effects on recognition performance. To address this problem, in this paper, a novel cross-corpus speech emotion recognition method, called transfer non-negative matrix factorization (TNMF) is proposed. Specifically, the NMF approach, which is popular in computer vision and pattern recognition fields, is utilized to obtain low dimensional representations of emotional features. Meanwhile, the discrepancies between source and target data sets are considered, and the maximum mean discrepancy (MMD) algorithm is used for similarity measurement. Then, the TNMF method, which jointly optimizes the NMF and MMD algorithms, is presented. Moreover, to further improve the recognition performance, two variants of TNMF, called transfer graph regularized NMF (TGNMF) and transfer constrained NMF (TCNMF), are proposed, respectively. Several experiments are carried out on three popular emotional databases, and the results demonstrate the effectiveness and robustness of our scheme.

Non-negative Matrix Tri-Factorization for co-clustering: An analysis of the block matrix

Non-negative dyadic data, that is data representing observations which relate two finite sets of objects, appear in several domain applications, such as text-mining-based information retrieval, collaborative filtering and recommender systems, micro-array analysis and computer vision. Discovering latent subgroups among data is a fundamental task to be performed on dyadic data. In this context, clustering and co-clustering techniques are relevant tools for extracting and representing latent information in high dimensional data. Recently, Non-negative Matrix Factorizations attracted a great interest as clustering methods, due to their capability of performing a parts-based decomposition of data. In this paper, we focus our attention on how NMF with additional constraints can be properly applied for co-clustering non-negative dyadic data. In particular, we present a process which aims at enhancing the performance of 3-factors NMF as a co-clustering method, by identifying a clearer correlation structure represented by the block matrix.Experimental evaluation performed on some common datasets, by applying the proposed approach on two different NMF algorithms, shows that, in most cases, the quality of the obtained clustering increases, especially in terms of average inter-cluster similarity.

Graph Regularized Nonnegative Matrix Factorization with Sparse Coding

In this paper, we propose a sparseness constraint NMF method, named graph regularized matrix factorization with sparse coding (GRNMF_SC). By combining manifold learning and sparse coding techniques together, GRNMF_SC can efficiently extract the basic vectors from the data space, which preserves the intrinsic manifold structure and also the local features of original data. The target function of our method is easy to propose, while the solving procedures are really nontrivial; in the paper we gave the detailed derivation of solving the target function and also a strict proof of its convergence, which is a key contribution of the paper. Compared with sparseness constrained NMF and GNMF algorithms, GRNMF_SC can learn much sparser representation of the data and can also preserve the geometrical structure of the data, which endow it with powerful discriminating ability. Furthermore, the GRNMF_SC is generalized as supervised and unsupervised models to meet different demands. Experimental results demonstrate encouraging results of GRNMF_SC on image recognition and clustering when comparing with the other state-of-the-art NMF methods.

Blind separation of analytes in nuclear magnetic resonance spectroscopy: Improved model for nonnegative matrix factorization

We introduce an improved model for sparseness-constrained nonnegative matrix factorization (sNMF) of amplitude nuclear magnetic resonance (NMR) spectra of mixtures into a greater number of component spectra. In the proposed method, the selected sNMF algorithm is applied to the square of the amplitude of the NMR spectrum of the mixture instead of to the amplitude spectrum itself. Afterwards, the square roots of separated squares of the component spectra and the concentration matrix yield estimates of the true component amplitude spectrum and of the concentration matrix. The proposed model remains linear on average when the number of overlapping components is increasing, while the model based on the amplitude spectra of the mixtures deviates from the linear one when the number of overlapping components is increased. This is demonstrated through the conducted sensitivity analysis. Thus, the proposed model improves the capability of the sparse NMF algorithms to separate correlated (overlapping) component spectra from the smaller number of mixture NMR spectra. This is demonstrated in two experimental scenarios: extraction of three correlated component spectra from two 1H NMR mixture spectra and extraction of four correlated component spectra from three COSY NMR mixture spectra. The proposed method can increase efficiency in a spectral library search by reducing the occurrence of false positives and false negatives. That, in turn, can yield better accuracy in biomarker identification studies, which makes the proposed method important for natural product research and the field of metabolic studies.

Double Constrained NMF for Hyperspectral Unmixing

Given only the collected hyperspectral data, unmixing aims at obtaining the latent constituent materials and their corresponding fractional abundances. Recently, many <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nonnegative matrix factorization</i> (NMF)-based algorithms have been developed to deal with this issue. Considering that the abundances of most materials may be sparse, the sparseness constraint is intuitively introduced into NMF. Although sparse NMF algorithms have achieved advanced performance in unmixing, the result is still susceptible to unstable decomposition and noise corruption. To reduce the aforementioned drawbacks, the structural information of the data is exploited to guide the unmixing. Since similar pixel spectra often imply similar substance constructions, clustering can explicitly characterize this similarity. Through maintaining the structural information during the unmixing, the resulting fractional abundances by the proposed algorithm can well coincide with the real distributions of constituent materials. Moreover, the additional clustering-based regularization term also lessens the interference of noise to some extent. The experimental results on synthetic and real hyperspectral data both illustrate the superiority of the proposed method compared with other state-of-the-art algorithms.

A new Bayesian approach to nonnegative matrix factorization: Uniqueness and model order selection

NMF is a blind source separation technique decomposing multivariate non-negative data sets into meaningful non-negative basis components and non-negative weights. There are still open problems to be solved: uniqueness and model order selection as well as developing efficient NMF algorithms for large scale problems. Addressing uniqueness issues, we propose a Bayesian optimality criterion (BOC) for NMF solutions which can be derived in the absence of prior knowledge. Furthermore, we present a new Variational Bayes NMF algorithm VBNMF which is a straight forward generalization of the canonical Lee–Seung method for the Euclidean NMF problem and demonstrate its ability to automatically detect the actual number of components in non-negative data.

Nonnegative Matrix Factorization Algorithms using a Constraint to Increase the Discriminability of Two Classes for EEG Feature Extraction

onnegative matrix factorization (NMF) is an algorithm used for blind source separation. It has been reported that NMF algorithms can be utilized as an effective means to extract features from a motor-imagery related EEG spectrum, which is often used in brain-computer interfaces (BCI). BCI systems enable users to control electrical devices without moving their body parts, and are often tasked with interpreting a user's intentions through motor-imagery related EEG features. In other words, they require EEG signal classification in order to reflect user intentions. In this study, constraints are placed on NMF and kernel NMF (KNMF) algorithms to increase the discriminability between two classes by increasing the energy difference between their potential sources in a spectral EEG signal. To evaluate the proposed algorithms, the IDIAP database, which contains the motor-imagery related EEG spectrum of three subjects, was adopted to test the discrimination between two classes. Using the database, the classification accuracy of the proposed constraint was 75%, which was 7% higher than what was obtained through NMF without a constraint. Similarly, the classification accuracy of KNMF with the proposed constraint was also 4% higher than that of KNMF without a constraint, and reached 78%.