Orthogonal Nonnegative Matrix Factorization Research Articles

An ordered subsets orthogonal nonnegative matrix factorization framework with application to image clustering

An ordered subsets orthogonal nonnegative matrix factorization framework with application to image clustering

Tensorized Soft Label Learning Based on Orthogonal NMF.

Recently, a strong interest has been in multiview high-dimensional data collected through cross-domain or various feature extraction mechanisms. Nonnegative matrix factorization (NMF) is an effective method for clustering these high-dimensional data with clear physical significance. However, existing multiview clustering based on NMF only measures the difference between the elements of the coefficient matrix without considering the spatial structure relationship between the elements. And they often require postprocessing to achieve clustering, making the algorithms unstable. To address this issue, we propose minimizing the Schatten p -norm of the tensor, which consists of a coefficient matrix of different views. This approach considers each element's spatial structure in the coefficient matrices, crucial for effectively capturing complementary information presented in different views. Furthermore, we apply orthogonal constraints to the cluster index matrix to make it sparse and provide a strong interpretation of the clustering. This allows us to obtain the cluster label directly without any postprocessing. To distinguish the importance of different views, we utilize adaptive weights to assign varying weights to each view. We introduce an unsupervised optimization scheme to solve and analyze the computational complexity of the model. Through comprehensive evaluations of six benchmark datasets and comparisons with several multiview clustering algorithms, we empirically demonstrate the superiority of our proposed method.

Detection of valvular heart diseases combining orthogonal non-negative matrix factorization and convolutional neural networks in PCG signals

Background and objective:Valvular heart disease (VHD) is associated with elevated mortality rates. Although transthoracic echocardiography (TTE) is the gold standard detection tool, phonocardiography (PCG) could be an alternative as it is a cost-effective and noninvasive method for cardiac auscultation. Many researchers have dedicated their efforts to improving the decision-making process and developing robust and precise approaches to assist physicians in providing reliable diagnoses of VHD. Methods:This research proposes a novel approach for the detection of anomalous valvular heart sounds from PCG signals. The proposed approach combines orthogonal non-negative matrix factorization (ONMF) and convolutional neural network (CNN) architectures in a three-stage cascade. The aim of the proposal is to improve the learning process by identifying the optimal ONMF temporal or spectral patterns for accurate detection. In the first stage, the time–frequency representation of the input PCG signal is computed. Next, band-pass filtering is performed to locate the spectral range that is most relevant for the presence of such cardiac abnormalities. In the second stage, the temporal and spectral cardiac structures are extracted using the ONMF approach. These structures are utilized in the third stage and fed into the CNN architecture to detect abnormal heart sounds. Results:Several state-of-the-art CNN architectures, such as LeNet5, AlexNet, ResNet50, VGG16 and GoogLeNet, have been evaluated to determine the effectiveness of using ONMF temporal features for VHD detection. The results reveal that the integration of ONMF temporal features with a CNN classifier significantly improve VHD detection. Specifically, the proposed approach achieves an accuracy improvement of approximately 45% when ONMF spectral features are used and 35% when time–frequency features from the short-time Fourier transform (STFT) spectrogram are used. Additionally, feeding ONMF temporal features into low-complexity CNN architectures yields competitive results comparable to those obtained with complex architectures. Conclusions:The temporal structure factorized by ONMF plays a critical role in distinguishing between normal heart sounds and abnormal heart sounds since the repeatability of normal heart cycles is disrupted by the presence of cardiac abnormalities. Consequently, the results highlight the importance of appropriate input data representation in the learning process of CNN models in the biomedical field of valvular heart sound detection.

Unilateral Orthogonal Nonnegative Matrix Factorization

Unilateral Orthogonal Nonnegative Matrix Factorization

HONMF: integration analysis of multi-omics microbiome data via matrix factorization and hypergraph.

The accumulation of multi-omics microbiome data provides an unprecedented opportunity to understand the diversity of bacterial, fungal, and viral components from different conditions. The changes in the composition of viruses, bacteria, and fungi communities have been associated with environments and critical illness. However, identifying and dissecting the heterogeneity of microbial samples and cross-kingdom interactions remains challenging. We propose HONMF for the integrative analysis of multi-modal microbiome data, including bacterial, fungal, and viral composition profiles. HONMF enables identification of microbial samples and data visualization, and also facilitates downstream analysis, including feature selection and cross-kingdom association analysis between species. HONMF is an unsupervised method based on hypergraph induced orthogonal non-negative matrix factorization, where it assumes that latent variables are specific for each composition profile and integrates the distinct sets of latent variables through graph fusion strategy, which better tackles the distinct characteristics in bacterial, fungal, and viral microbiome. We implemented HONMF on several multi-omics microbiome datasets from different environments and tissues. The experimental results demonstrate the superior performance of HONMF in data visualization and clustering. HONMF also provides rich biological insights by implementing discriminative microbial feature selection and bacterium-fungus-virus association analysis, which improves our understanding of ecological interactions and microbial pathogenesis. The software and datasets are available at https://github.com/chonghua-1983/HONMF.

Formula omitted]-divergence NMF with biorthogonal regularization for data representation

Non-Negative Matrix Factorization (NMF) has become a commonly used method for data representation. Orthogonal NMF improves the clustering performance by adding orthogonal constraints to the decomposed matrices. The existing orthogonal NMF methods typically use Euclidean distance to measure the difference between before and after factorization for convenience and simplicity. However, limitations of the Euclidean distance can lead to inflexibilities. In addition, failure to consider orthogonality of the decomposed features and sparsity of the data representation can also lead to degraded performance of the algorithm. In order to overcome the above shortcomings, we propose a novel β-divergence-based NMF with biorthogonal regularization (BO-βNMF). Our BO-βNMF method uses generalized β-divergence instead of Euclidean distance to measure the similarity between matrices, and selects an appropriate β for each type of data to obtain a more flexible way of measuring similarity. In addition, we also incorporate biorthogonal constraints into the minimized objective function, which ensures both orthogonality of the decomposed features and sparsity of the data representation. Furthermore, we use trace rather than Euclidean distance to measure the orthogonality of the decomposed matrices, which reduces execution time. Finally, clustering experiments on image datasets show that the overall clustering effect of BO-βNMF is better than state-of-the-art methods.

Identification of the Subtypes of Renal Ischemia-Reperfusion Injury Based on Pyroptosis-Related Genes.

Ischemia-reperfusion injury (IRI) often occurs in the process of kidney transplantation, which significantly impacts the subsequent treatment and prognosis of patients. The prognosis of patients with different subtypes of IRI is quite different. Therefore, in this paper, the gene expression data of multiple IRI samples were downloaded from the GEO database, and a double Laplacian orthogonal non-negative matrix factorization (DL-ONMF) algorithm was proposed to classify them. In this algorithm, various regularization constraints are added based on the non-negative matrix factorization algorithm, and the prior information is fused into the algorithm from different perspectives. The connectivity information between different samples and features is added to the algorithm by Laplacian regularization constraints on samples and features. In addition, orthogonality constraints on the basis matrix and coefficient matrix obtained by the algorithm decomposition are added to reduce the influence of redundant samples and redundant features on the results. Based on the DL-ONMF algorithm for clustering, two PRGs-related IRI isoforms were obtained in this paper. The results of immunoassays showed that the immune microenvironment was different among PRGS-related IRI types. Based on the differentially expressed PRGs between subtypes, we used LASSO and SVM-RFE algorithms to construct a diagnostic model related to renal transplantation. ROC analysis showed that the diagnostic model could predict the outcome of renal transplant patients with high accuracy. In conclusion, this paper presents an algorithm, DL-ONMF, which can identify subtypes with different disease characteristics. Comprehensive bioinformatic analysis showed that pyroptosis might affect the outcome of kidney transplantation by participating in the immune response of IRI.

Structured Joint Sparse Orthogonal Nonnegative Matrix Factorization for Fault Detection

As modern industrial processes become complicated, and some faults are difficult to be detected due to noises and nonlinearity of data, data-driven fault detection has been extensively used to detect abnormal events in functional units. In order to obtain the better fault detection performance of nonnegative matrix factorization, this paper first proposes a fault detection method using the structured joint sparse orthogonal nonnegative matrix factorization. The core idea is to incorporate the graph regularization, sparsity and orthogonality constraints into the classical nonnegative matrix factorization, which enjoys stronger discriminative ability, removes redundancy of different basis vectors and improves the fault interpretability. More importantly, an optimization algorithm based on the proximal alternating nonnegative least squares is developed, which can guarantee and speed up the convergence. Finally, the effectiveness of the proposed method is demonstrated by the experiments on the benchmark Tennessee Eastman Process and two practical bearing datasets. Particularly, compared with the classical nonnegative matrix factorization, the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> statistic has a gain of 33.13% for the fault IDV(16) on the Tennessee Eastman Process. The results show that the proposed model and algorithms are promising for the fault detection.

A majorization–minimization based solution to penalized nonnegative matrix factorization with orthogonal regularization

Nonnegative matrix factorization (NMF) is a dimension reduction and clustering technique for data analysis which has been widely used in image processing, text analysis and hyperspectral decomposition because of its stronger practical significance and better interpretability. Approximate matrix factorization techniques with both nonnegativity and orthogonality constraints, referred to as orthogonal NMF (ONMF), have been shown to work remarkably better for clustering tasks than NMF. At present, a large number of algorithms have been used to solve the ONMF problems, but these methods usually cannot take into account the classification accuracy and calculation speed. In this paper, we propose a new form of penalized NMF with orthogonal regularization that combines the decomposition residual minimization based on the Euclidean distance and the orthogonality maximization based on the Kullback–Leibler divergence. This paper uses Majorization–Minimization (MM) method by minimizing a majorization function of the original problem and obtains a new iterative scheme (MM-ONMF). Comparing with several traditional ONMF methods on eight datasets, experimental results show that the proposed method has better clustering results and less computing time.

Co-consensus semi-supervised multi-view learning with orthogonal non-negative matrix factorization

Semi-supervised multi-view learning has recently achieved appealing performance with the consensus relation between samples. However, in addition to the relation between samples, the relation between samples and their assemble centroid is also important to the learning. In this paper, we propose a novel model based on orthogonal non-negative matrix factorization, which allows exploring both the consensus relations between samples and between samples and their assemble centroid. Since this model utilizes more consensus information to guide the multi-view learning, it can lead to better performance. Meanwhile, we theoretically derive a proposition about the equivalency between the partial orthogonality and the full orthogonality. Based on this proposition, the orthogonality constraint and the label constraint are simultaneously implemented in the proposed model. Experimental evaluations on five real-world datasets show that our approach outperforms the state-of-the-art methods, where the improvement is 6% average in terms of ARI index.

Incomplete multi-view clustering with incomplete graph-regularized orthogonal non-negative matrix factorization

Incomplete multi-view clustering with incomplete graph-regularized orthogonal non-negative matrix factorization

Ridge count thresholding to uncover coordinated networks during onset of the Covid-19 pandemic.

In order to combat information operations (IO) and disinformation campaigns, one must look at the behaviors of the accounts pushing specific narratives and stories through social media, not at the content itself. In this work, we present a new process for extracting tweet storms and uncovering networks of accounts that are working in a coordinated fashion using ridge count thresholding (RCT). To do this, we started with a dataset of 60 million individual tweets from the early weeks of the Covid-19 pandemic. Coherent topics are extracted from this data by testing three different preprocessing pipelines and applying Orthogonal Nonnegative Matrix Factorization (ONMF). The most effective preprocessing pipeline used hashtag preclustering to downselect the total dataset to the 7 million tweets that included the top hashtags. Each topic identified by ONMF is described by a topic-tweet signal, crafted using the time stamp included in each tweet’s metadata. These signals were broken down into tweet storms using RCT, which is calculated from the Dynamic Wavelet Fingerprint transform of each topic-tweet signal. Each tweet storm described a time of increased activity around a topic. Tweet storms identified in this way each represent some behavior in the underlying network. In total, we identified 39,817 total tweet storms that included about 2 million unique tweets. These tweet storms were used to identify networks of accounts that commonly co-occur within tweet storms to isolate those communities most responsible for driving narratives and pushing stories through social media. Through this process, we were able to identify 22 unique networks of accounts that were densely connected based on RCT tweet storm identification. Many of the identified networks exhibit obvious inauthentic behaviors that are potentially a part of an IO campaign.

An exact penalty approach for optimization with nonnegative orthogonality constraints

Optimization with nonnegative orthogonality constraints has wide applications in machine learning and data sciences. It is NP-hard due to some combinatorial properties of the constraints. We first propose an equivalent optimization formulation with nonnegative and multiple spherical constraints and an additional single nonlinear constraint. Various constraint qualifications, the first- and second-order optimality conditions of the equivalent formulation are discussed. By establishing a local error bound of the feasible set, we design a class of (smooth) exact penalty models via keeping the nonnegative and multiple spherical constraints. The penalty models are exact if the penalty parameter is sufficiently large but finite. A practical penalty algorithm with postprocessing is then developed to approximately solve a series of subproblems with nonnegative and multiple spherical constraints. We study the asymptotic convergence and establish that any limit point is a weakly stationary point of the original problem and becomes a stationary point under some additional mild conditions. Extensive numerical results on the problem of computing the orthogonal projection onto nonnegative orthogonality constraints, the orthogonal nonnegative matrix factorization problems and the K-indicators model show the effectiveness of our proposed approach.

Orthogonal nonnegative matrix factorization problems for clustering: A new formulation and a competitive algorithm

Orthogonal nonnegative matrix factorization problems for clustering: A new formulation and a competitive algorithm

Block Bregman Majorization Minimization with Extrapolation

In this paper, we consider a class of nonsmooth nonconvex optimization problems whose objective is the sum of a block relative smooth function and a proper and lower semicontinuous block separable function. Although the analysis of block proximal gradient (BPG) methods for the class of block $L$-smooth functions have been successfully extended to Bregman BPG methods that deal with the class of block relative smooth functions, accelerated Bregman BPG methods are scarce and challenging to design. Taking our inspiration from Nesterov-type acceleration and the majorization-minimization scheme, we propose a block alternating Bregman Majorization-Minimization framework with Extrapolation (BMME). We prove subsequential convergence of BMME to a first-order stationary point under mild assumptions, and study its global convergence under stronger conditions. We illustrate the effectiveness of BMME on the penalized orthogonal nonnegative matrix factorization problem.

Variational Bayesian Orthogonal Nonnegative Matrix Factorization Over the Stiefel Manifold.

Nonnegative matrix factorization (NMF) is one of the best-known multivariate data analysis techniques. The NMF uniqueness and its rank selection are two major open problems in this field. The solutions uniqueness issue can be addressed by imposing the orthogonality condition on NMF. This constraint yields sparser part-based representations and improved performance in clustering and source separation tasks. However, existing orthogonal NMF algorithms rely mainly on non-probabilistic frameworks that ignore the noise inherent in real-life data and lack variable uncertainties. Thus, in this work, we investigate a new probabilistic formulation of orthogonal NMF (ONMF). In the proposed model, we impose the orthogonality through a directional prior distribution defined on the Stiefel manifold called von Mises-Fisher distribution. This manifold consists of a set of directions that comply with the orthogonality condition that arises in many applications. Moreover, our model involves an automatic relevance determination (ARD) prior to address the model order selection issue. We devised an efficient variational Bayesian inference algorithm to solve the proposed ONMF model, which allows fast processing of large datasets. We evaluated the proposed model, called VBONMF, on the task of blind decomposition of real-world multispectral images of ancient documents. The numerical experiments demonstrate its efficiency and competitiveness compared to the state-of-the-art approaches.

Non-negative matrix factorization for 2D-XAS images of lithium ion batteries

Lithium-ion secondary batteries have been used in a wide variety of purposes, such as for powering mobile devices and electric vehicles, but their performance should be improved. One of the factors that limits their performance is the non-uniformity of the chemical reaction in the process of charging and discharging. Many attempts have been made to elucidate the mechanism behind this reaction non-uniformity. In this paper, to detect non-uniformity in various physical properties from Co K-edge two-dimensional x-ray absorption spectroscopy (2D-XAS) images of lithium ion batteries, we propose a method that consists of one-sided orthogonal non-negative matrix factorization in combination with removal of the reference signal. The difference between x-ray absorption spectra acquired at different positions in the battery is very small. However, even in such a situation, our method can decompose the 2D-XAS data into different spatial domains and their corresponding absorption spectra. From the spectral decomposition of the obtained absorption spectra, we confirmed a transition-energy shift of the main peak as evidence for a change in the state of charge and also found spectral changes due to orbital hybridization in the decomposed spectral components.

Spatially Coherent Clustering Based on Orthogonal Nonnegative Matrix Factorization.

Classical approaches in cluster analysis are typically based on a feature space analysis. However, many applications lead to datasets with additional spatial information and a ground truth with spatially coherent classes, which will not necessarily be reconstructed well by standard clustering methods. Motivated by applications in hyperspectral imaging, we introduce in this work clustering models based on Orthogonal Nonnegative Matrix Factorization (ONMF), which include an additional Total Variation (TV) regularization procedure on the cluster membership matrix to enforce the needed spatial coherence in the clusters. We propose several approaches with different optimization techniques, where the TV regularization is either performed as a subsequent post-processing step or included into the clustering algorithm. Finally, we provide a numerical evaluation of 12 different TV regularized ONMF methods on a hyperspectral dataset obtained from a matrix-assisted laser desorption/ionization imaging measurement, which leads to significantly better clustering results compared to classical clustering models.

Two Efficient Algorithms for Orthogonal Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is a popular method for the multivariate analysis of nonnegative data. It involves decomposing a data matrix into a product of two factor matrices with all entries restricted to being nonnegative. Orthogonal nonnegative matrix factorization (ONMF) has been introduced recently. This method has demonstrated remarkable performance in clustering tasks, such as gene expression classification. In this study, we introduce two convergence methods for solving ONMF. First, we design a convergent orthogonal algorithm based on the Lagrange multiplier method. Second, we propose an approach that is based on the alternating direction method. Finally, we demonstrate that the two proposed approaches tend to deliver higher-quality solutions and perform better in clustering tasks compared with a state-of-the-art ONMF.

Orthogonal Nonnegative Matrix Factorization using a novel deep Autoencoder Network

Orthogonal Nonnegative Matrix Factorization (ONMF) offers an important analytical vehicle for addressing many problems. Encouraged by record-breaking successes attained by neural computing models in solving an assortment of data analytics tasks, a rich collection of neural computing models has been proposed to perform ONMF with compelling performance. Such existing models can be broadly classified into the shallow-layered structure (SLS) based and deep-layered structure (DLS) based models. However, SLS models cannot capture complex relationships and hierarchical information latent in a matrix due to their simple network structures and DLS models rely on an iterative procedure to derive weights, leading to a less efficient solution process and cannot be reused to factorize new matrices. To overcome these shortcomings, this paper proposes a novel deep autoencoder network for ONMF, which is abbreviated as DAutoED-ONMF. Compared with SLS models, the newly proposed model is capable of generating solutions with good interpretability and solution uniqueness like original SLS models, yet the new model attains a superior learning capability thanks to its deep structure employed. In comparison with DLS models, the new model trains a reusable encoder network to directly factorize any given matrix with no need to repeatedly retrain the model for factorizing multiple matrices using a tailor-designed network training procedure. Proof of the procedure’s convergence is presented with an analysis of its computational complexity. The numerical experiments conducted on several publicly data sets convincingly demonstrate that the proposed DAutoED-ONMF model gains promising performance in terms of multiple metrics.