Probabilistic Non-negative Matrix Factorization Research Articles

Interpretable machine learning model using Extreme Gradient Boosting with Nonnegative Matrix Factorization improves the accuracy of arrhythmic risk prediction in Brugada Syndrome

Abstract Background Both depolarization and repolarization abnormalities contribute to ventricular arrhythmogenesis in Brugada syndrome (BrS). In this study, we tested the hypothesis that incorporating latent features extracted by various nonnegative matrix factorization (NML) techniques into an interpretable machine learning (IML) prediction model can outperform IML models without latent variables and logistic regression model. Methods This study was based on a published anonymised dataset of BrS patients from the Hong Kong, China. XGBoost was selected as the IML model, with and without incorporating latent features using 11 different NMF techniques: Bayesian nonnegative matrix factorization (BNMF), Iterated Conditional Modes nonnegative matrix factorization (ICM), Fisher Nonnegative Matrix Factorization for learning Local features (LFNMF), Alternating Nonnegative Least Squares Matrix Factorization Using Projected Gradient (bound constrained optimization) method for each subproblem (LSNMF), Non-smooth Nonnegative Matrix Factorization (NSNMF), Probabilistic Nonnegative Matrix Factorization (PMF), Probabilistic Sparse Matrix Factorization (PSMF), Sparse Nonnegative Matrix Factorization (SNMF) based on alternating nonnegativity constrained least squares, Sparse Network-Regularized Multiple Nonnegative Matrix Factorization (SNMNMF), Penalized Matrix Factorization for Constrained Clustering (PMFCC) and Separable Nonnegative Matrix Factorization (SepNMF). Results A total of 548 patients were included (7.3% females, age at diagnosis: 51.0 [38.0-61.0] years old. Of these, 66 suffered from spontaneous ventricular tachyarrhythmias over 84±55 months follow-up. The baseline model using multivariable logistic regression achieved an area under the curve (AUC) of 0.78 [0.72-0.85], which was improved to 0.88 [0.83-0.93] for the IML model without NMF. The AUC was further increased by incorporating additional latent variables extracted using NSNMF (0.95 [0.92-0.98]) and PMFCC (0.94 [0.88-1.00]). Conclusion Incorporation of latent variables by different NMF techniques into an IML prediction model significantly improved the accuracy of risk prediction in BrS.

Nonnegative spatial factorization applied to spatial genomics

Nonnegative matrix factorization (NMF) is widely used to analyze high-dimensional count data because, in contrast to real-valued alternatives such as factor analysis, it produces an interpretable parts-based representation. However, in applications such as spatial transcriptomics, NMF fails to incorporate known structure between observations. Here, we present nonnegative spatial factorization (NSF), a spatially-aware probabilistic dimension reduction model based on transformed Gaussian processes that naturally encourages sparsity and scales to tens of thousands of observations. NSF recovers ground truth factors more accurately than real-valued alternatives such as MEFISTO in simulations, and has lower out-of-sample prediction error than probabilistic NMF on three spatial transcriptomics datasets from mouse brain and liver. Since not all patterns of gene expression have spatial correlations, we also propose a hybrid extension of NSF that combines spatial and nonspatial components, enabling quantification of spatial importance for both observations and features. A TensorFlow implementation of NSF is available from https://github.com/willtownes/nsf-paper.

Algorithms for audio inpainting based on probabilistic nonnegative matrix factorization

Audio inpainting, i.e., the task of restoring missing or occluded audio signal samples, usually relies on sparse representations or autoregressive modeling. In this paper, we propose to structure the spectrogram with nonnegative matrix factorization (NMF) in a probabilistic framework. First, we treat the missing samples as latent variables, and derive two expectation–maximization algorithms for estimating the parameters of the model, depending on whether we formulate the problem in the time- or time-frequency domain. Then, we treat the missing samples as parameters, and we address this novel problem by deriving an alternating minimization scheme. We assess the potential of these algorithms for the task of restoring short- to middle-length gaps in music signals. Experiments reveal great convergence properties of the proposed methods, as well as competitive performance when compared to state-of-the-art audio inpainting techniques.

Link Prediction Under Imperfect Detection: Collaborative Filtering for Ecological Networks

Matrix completion based collaborative filtering is considered scalable and effective for online service link prediction (e.g., movie recommendation) but does not meet the challenges of link prediction in ecological networks. A unique challenge of ecological networks is that the observed data are subject to systematic imperfect detection, due to the difficulty of accurate field sampling. In this work, we propose a new framework customized for ecological bipartite network link prediction. Our approach starts with incorporating the Poisson N-mixture model, a widely used framework in statistical ecology for modeling imperfect detection of a single species in field sampling. Despite its extensive use for single species analysis, this model has never been considered for link prediction between different species, perhaps because of the complex nature of both link prediction and N-mixture model inference. By judiciously combining the Poisson N-mixture model with a probabilistic nonnegative matrix factorization (NMF) model in latent space, we propose an intuitive statistical model for the problem of interest. We also offer a scalable and convergence-guaranteed optimization algorithm to handle the associated maximum likelihood identification problem. Experimental results on synthetic data and two real-world ecological networks data are employed to validate our proposed approach.

Probabilistic Non-Negative Matrix Factorization with Binary Components

Non-negative matrix factorization is used to find a basic matrix and a weight matrix to approximate the non-negative matrix. It has proven to be a powerful low-rank decomposition technique for non-negative multivariate data. However, its performance largely depends on the assumption of a fixed number of features. This work proposes a new probabilistic non-negative matrix factorization which factorizes a non-negative matrix into a low-rank factor matrix with constraints and a non-negative weight matrix. In order to automatically learn the potential binary features and feature number, a deterministic Indian buffet process variational inference is introduced to obtain the binary factor matrix. Further, the weight matrix is set to satisfy the exponential prior. To obtain the real posterior distribution of the two factor matrices, a variational Bayesian exponential Gaussian inference model is established. The comparative experiments on the synthetic and real-world datasets show the efficacy of the proposed method.

A Comparative Study of Gamma Markov Chains for Temporal Non-Negative Matrix Factorization

Non-negative matrix factorization (NMF) has become a well-established class of methods for the analysis of non-negative data. In particular, a lot of effort has been devoted to probabilistic NMF, namely estimation or inference tasks in probabilistic models describing the data, based for example on Poisson or exponential likelihoods. When dealing with time series data, several works have proposed to model the evolution of the activation coefficients as a non-negative Markov chain, most of the time in relation with the Gamma distribution, giving rise to so-called temporal NMF models. In this paper, we review four Gamma Markov chains of the NMF literature, and show that they all share the same drawback: the absence of a well-defined stationary distribution. We then introduce a fifth process, an overlooked model of the time series literature named BGAR(1), which overcomes this limitation. These temporal NMF models are then compared in a MAP framework on a prediction task, in the context of the Poisson likelihood.

Bayesian mean-parameterized nonnegative binary matrix factorization

Binary data matrices can represent many types of data such as social networks, votes, or gene expression. In some cases, the analysis of binary matrices can be tackled with nonnegative matrix factorization (NMF), where the observed data matrix is approximated by the product of two smaller nonnegative matrices. In this context, probabilistic NMF assumes a generative model where the data is usually Bernoulli-distributed. Often, a link function is used to map the factorization to the [0, 1] range, ensuring a valid Bernoulli mean parameter. However, link functions have the potential disadvantage to lead to uninterpretable models. Mean-parameterized NMF, on the contrary, overcomes this problem. We propose a unified framework for Bayesian mean-parameterized nonnegative binary matrix factorization models (NBMF). We analyze three models which correspond to three possible constraints that respect the mean-parameterization without the need for link functions. Furthermore, we derive a novel collapsed Gibbs sampler and a collapsed variational algorithm to infer the posterior distribution of the factors. Next, we extend the proposed models to a nonparametric setting where the number of used latent dimensions is automatically driven by the observed data. We analyze the performance of our NBMF methods in multiple datasets for different tasks such as dictionary learning and prediction of missing data. Experiments show that our methods provide similar or superior results than the state of the art, while automatically detecting the number of relevant components.

Probabilistic Non-Negative Matrix Factorization and Its Robust Extensions for Topic Modeling

Traditional topic model with maximum likelihood estimate inevitably suffers from the conditional independence of words given the document’s topic distribution. In this paper, we follow the generative procedure of topic model and learn the topic-word distribution and topics distribution via directly approximating the word-document co-occurrence matrix with matrix decomposition technique. These methods include: (1) Approximating the normalized document-word conditional distribution with the documents probability matrix and words probability matrix based on probabilistic non-negative matrix factorization (NMF); (2) Since the standard NMF is well known to be non-robust to noises and outliers, we extended the probabilistic NMF of the topic model to its robust versions using l21-norm and capped l21-norm based loss functions, respectively. The proposed framework inherits the explicit probabilistic meaning of factors in topic models and simultaneously makes the conditional independence assumption on words unnecessary. Straightforward and efficient algorithms are exploited to solve the corresponding non-smooth and non-convex problems. Experimental results over several benchmark datasets illustrate the effectiveness and superiority of the proposed methods.

확률적 비음수 행렬 인수분해를 사용한 통계적 음성검출기법

본 논문은 비음수 행렬 인수분해(NMF)의 확률적 해석에 근거한 새로운 통계적 음성검출기법을 제안한다. NMF의 기저와 부호화 행렬들이 주어졌을 때, 데이터 행렬의 분포를 Poisson 분포로 가정한 로그 우도는 Kullback-Leibler 발산을 이용한 NMF의 목적 함수와 일치한다. 이러한 NMF의 확률모델에 근거하여 음성검출을 위해 DFT영역에서 잡음과 음성의 크기 스펙트럼을 Poisson 분포로 모델링하여 새로운 우도비 검출 규칙을 유도한다. 실험 결과를 통해 제안된 기법이 0-15dB 신호 대 잡음비의 시뮬레이션 환경에서 기존 Gaussian과 NMF을 사용한 기법보다 향상된 음성검출 결과를 보여준다. This paper presents a new statistical voice activity detection (VAD) based on the probabilistic interpretation of nonnegative matrix factorization (NMF). The objective function of the NMF using Kullback-Leibler divergence coincides with the negative log likelihood function of the data if the distribution of the data given the basis and encoding matrices is modeled as Poisson distributions. Based on this probabilistic NMF, the VAD is constructed using the likelihood ratio test assuming that speech and noise follow Poisson distributions. Experimental results show that the proposed approach outperformed the conventional Gaussian model-based and NMF-based methods at 0-15 dB signal-to-noise ratio simulation conditions.

Local quality functions for graph clustering with non-negative matrix factorization.

Many graph clustering quality functions suffer from a resolution limit, namely the inability to find small clusters in large graphs. So-called resolution-limit-free quality functions do not have this limit. This property was previously introduced for hard clustering, that is, graph partitioning. We investigate the resolution-limit-free property in the context of non-negative matrix factorization (NMF) for hard and soft graph clustering. To use NMF in the hard clustering setting, a common approach is to assign each node to its highest membership cluster. We show that in this case symmetric NMF is not resolution-limit free, but that it becomes so when hardness constraints are used as part of the optimization. The resulting function is strongly linked to the constant Potts model. In soft clustering, nodes can belong to more than one cluster, with varying degrees of membership. In this setting resolution-limit free turns out to be too strong a property. Therefore we introduce locality, which roughly states that changing one part of the graph does not affect the clustering of other parts of the graph. We argue that this is a desirable property, provide conditions under which NMF quality functions are local, and propose a novel class of local probabilistic NMF quality functions for soft graph clustering.

Non-informative hierarchical Bayesian inference for non-negative matrix factorization

Non-negative matrix factorization (NMF) is an intuitive, non-negative, and interpretable approximation method. Canonical NMF approach could derive some basic components to represent original data, while probabilistic NMF approaches try to introduce some reasonable constraints to optimize the canonical NMF model. However, both of them cannot handle ground-truth bases discovering and model order determination problems. In general, the model order of basis matrix needs to be pre-defined. The model order determines the capability and accuracy of data structure discovering. However, how to accurately infer the model order of basis matrix has not been well investigated. In this paper, we propose a method called non-informative hierarchical Bayesian non-negative matrix factorization (NHBNMF) to automatically determine the model order and discover the data structure. They are achieved through hierarchical Bayesian inference model, maximum a posteriori (MAP) criterion, and non-informative parameters. In NHBNMF method, we first introduce a structure with two-level parameters to enable the entire model to approach the distributions of ground-truth bases. Then we use non-informative parameter scheme to eliminate the hyper-parameter to enable automatic searching. Finally, the model order and ground-truth bases are discovered by using MAP criterion and L2-norm selection. The experiments are conducted based on both synthetic and real-world datasets to show the effectiveness of our algorithm. The results demonstrate that our algorithm can accurately estimate the model order and discover the ground-truth bases. Even for the complicated FERET facial dataset, our algorithm still obtained interpretable bases and achieved satisfactory accuracy of the model order estimation.

Mobile Target Scenario Recognition Via Low-Cost Pyroelectric Sensing System: Toward a Context-Enhanced Accurate Identification

Distributed binary pyroelectric sensor network (PSN) is a low-cost alternative to video systems for human monitoring applications. This paper presents a PSN-based mobile target recognition system, which aims to achieve multitarget, complex scenario recognition. In this system, a novel pseudorandom visibility mode is designed for the sensor arrays to help capture statistical information of scenarios, and a sensor array fusion scheme is adopted to facilitate discriminative feature extraction. Moreover, we propose a statistical subspace representation model called probabilistic nonnegative matrix factorization (PNMF) to seek the scenario patterns rather than the object characteristics. We also further prove that our PNMF model is a generic model for NMF based algorithms. Original NMF, sparse NMF, and smooth NMF are special cases of the PNMF model. The simulation and experimental results demonstrate the advantages of our proposed method. Our system can be further developed to function as an independent facility for intelligent monitoring applications, especially under poor illumination circumstances.