Basic Non-negative Matrix Factorization Research Articles

Micro-Doppler Extraction for Cone-Shaped Target Using Constrained Nonnegative Matrix Factorization

Micro-Doppler (MD) extraction is the precondition for target classification based on micro-motion. This letter proposes a method for MD extraction based on constrained nonnegative matrix factorization (NMF). The proposed algorithm is applied to the time-frequency spectrogram of the cone-shaped precession target. Taking the sparsity constraint, temporal continuity constraint and approximate orthogonality constraint into consideration, the basic NMF is improved to match the MD separation problem, and thus the time-frequency spectrogram of each scattering center is reconstructed by deriving the basis matrix and coefficient matrix. Morphological processing, valid frequency extraction and MD curve fitting are conducted on the reconstructed spectrograms to extract the MD accurately. Experiment results show that the proposed approach outperforms other algorithms with better accuracy and robustness.

Automatic Extractive and Generic Document Summarization Based on NMF

Abstract Nowadays, textual information grows exponentially on the Internet. Text summarization (TS) plays a crucial role in the massive amount of textual content. Manual TS is time-consuming and impractical in some applications with a huge amount of textual information. Automatic text summarization (ATS) is an essential technology to overcome mentioned challenges. Non-negative matrix factorization (NMF) is a useful tool for extracting semantic contents from textual data. Existing NMF approaches only focus on how factorized matrices should be modeled, and neglect the relationships among sentences. These relationships provide better factorization for TS. This paper suggests a novel non-negative matrix factorization for text summarization (NMFTS). The proposed ATS model puts regularizes on pairwise sentences vectors. A new cost function based on the Frobenius norm is designed, and an algorithm is developed to minimize this function by proposing iterative updating rules. The proposed NMFTS extracts semantic content by reducing the size of documents and mapping the same sentences closely together in the latent topic space. Compared with the basic NMF, the convergence time of the proposed method does not grow. The convergence proof of the NMFTS and empirical results on the benchmark data sets show that the suggested updating rules converge fast and achieve superior results compared to other methods.

Deep NMF topic modeling

Nonnegative matrix factorization (NMF) based topic modeling methods do not rely on model- or data-assumptions much. However, they are usually formulated as difficult optimization problems, which may suffer from bad local minima and high computational complexity. In this paper, we propose a deep NMF (DNMF) topic modeling framework to alleviate the aforementioned problems. It first applies an unsupervised deep learning method to learn latent hierarchical structures of documents, under the assumption that if we could learn a good representation of documents by, e.g. a deep model, then the topic word discovery problem can be boosted. Then, it takes the output of the deep model to constrain a topic-document distribution for the discovery of the discriminant topic words, which not only improves the efficacy but also reduces the computational complexity over conventional unsupervised NMF methods. We constrain the topic-document distribution in three ways, which takes the advantages of the three major sub-categories of NMF—basic NMF, structured NMF, and constrained NMF respectively. To overcome the weaknesses of deep neural networks in unsupervised topic modeling, we adopt a non-neural-network deep model—multilayer bootstrap network. To our knowledge, this is the first time that a deep NMF model is used for unsupervised topic modeling. We have compared the proposed method with a number of representative references covering major branches of topic modeling on a variety of real-world text corpora. Experimental results illustrate the effectiveness of the proposed method under various evaluation metrics.

A novel constrained non-negative matrix factorization method based on users and items pairwise relationship for recommender systems

Non-negative matrix factorization (NMF) is a famous method to learn parts-based representations of non-negative data. It has been used successfully in various applications such as information retrieval and recommender systems. Most of the current NMF methods only focus on how each decomposed matrices vector should be modeled and disregard the relation among pairwise feature vectors of users and items. In recommender systems, the relationship between users’ factors or items’ features provides better factorization. This paper proposes a constrained non-negative matrix factorization for recommender systems (CNMF-RS), which puts constraints on pairwise vectors of users and items by applying penalties using similarity. Two cost functions are proposed to generate the corresponding updating rules for finding unknown ratings. The computational complexity analysis confirmed that CNMF-RS does not raise time complexity using constraints compared with the basic NMF. Experiments on benchmark datasets indicate that the suggested update rules converge quickly and obtain outstanding results compared to other methods.

Clustering US States by Time Series of COVID-19 New Case Counts in the Early Months with Non-Negative Matrix Factorization

The spreading pattern of COVID-19 in the early months of the pandemic differs a lot across the states in the US under different quarantine measures and reopening policies. We proposed to cluster the US states into distinct communities based on the daily new confirmed case counts from March 22 to July 25 via a nonnegative matrix factorization (NMF) followed by a k-means clustering procedure on the coefficients of the NMF basis. A cross-validation method was employed to select the rank of the NMF. The method clustered the 49 continental states (including the District of Columbia) into 7 groups, two of which contained a single state. To investigate the dynamics of the clustering results over time, the same method was successively applied to the time periods with an increment of one week, starting from the period of March 22 to March 28. The results suggested a change point in the clustering in the week starting on May 30, caused by a combined impact of both quarantine measures and reopening policies.

A supervised non-negative matrix factorization model for speech emotion recognition

Feature representation plays a critical role in speech emotion recognition (SER). As a method of data dimensionality reduction, Non-negative Matrix Factorization (NMF) can obtain the low-dimensional representation of data by matrix decomposition, and make the data more distinguishable. In order to improve the recognition ability of NMF for SER, we conduct a potential study on NMF and propose a supervised NMF model, called joint discrimination ability and similarity constraint of NMF (DSNMF). This model incorporates the discriminative information and similarity information of samples into basic NMF as prior knowledge, so that the original data can be decomposed into more distinguished low-dimensional data. Specifically, on the one hand, the labels of the training set are used to improve the discriminative ability of the model; on the other hand, with the similarity of the training samples, the data of similar samples are more highly aggregated in the low-dimensional space. In addition, the convergence of DSNMF is proved theoretically and experimentally. Extensive experiments on EMODB and IEMOCAP corpora show that the proposed approach has a better classification effect on low-dimensional representation data than other NMF models.

Statistical 3D watermarking algorithm using non negative matrix factorization

In this paper, we propose a robust blind watermarking method for 3D object based on Non Negative Matrix Factorization (NMF). Our main idea is to use the NMF basis matrix guide the watermark embedding. We first segment the model into partitions based on its salient points. Then, we apply NMF algorithm to the vertex norms. The distribution of vertex is grouped into bins according to the basis matrix X. Finally, we insert watermark bits in the vertex norm distribution. Experimental results demonstrate that the proposed method achieved good visual quality and also robustness against various attacks.

ShinyButchR: Interactive NMF-based decomposition workflow of genome-scale datasets.

Non-negative matrix factorization (NMF) has been widely used for the analysis of genomic data to perform feature extraction and signature identification due to the interpretability of the decomposed signatures. However, running a basic NMF analysis requires the installation of multiple tools and dependencies, along with a steep learning curve and computing time. To mitigate such obstacles, we developed ShinyButchR, a novel R/Shiny application that provides a complete NMF-based analysis workflow, allowing the user to perform matrix decomposition using NMF, feature extraction, interactive visualization, relevant signature identification, and association to biological and clinical variables. ShinyButchR builds upon the also novel R package ButchR, which provides new TensorFlow solvers for algorithms of the NMF family, functions for downstream analysis, a rational method to determine the optimal factorization rank and a novel feature selection strategy.

Nonnegative matrix factorization for the identification of pressure ulcer risks from seating interface pressures in people with spinal cord injury.

The purpose of this study was to predict and visualize pressure ulcer risks by using a novel approach of extracting computational features from seating interface pressures in people with spinal cord injury (SCI). In conventional clinical practice, seating interface pressure assessments rely on descriptive statistics of pressure magnitude. In this study, rank-2 nonnegative matrix factorization (NMF) was applied to the seating interface pressure maps during loading and pressure-relieving conditions in 16 people with SCI. The NMF basis images were used for visual interpretation and computational prediction of pressure ulcer risks. The two NMF basis images encapsulated pressure concentration and pressure dispersion, respectively. The first basis converged on the ischial tuberosity under both seating conditions, whereas the second basis converged anterior to the ischial tuberosity during loading and converged on the coccyx during unloading. The classification yielded 81.25% overall accuracy. In general, higher ulceration risk was associated with higher and lower activations of the first and second bases, respectively. The NMF pipeline yielded promising performance. Basis visualization affirmed the importance of lower ischial pressure and higher distribution dispersion while also revealing that clinical practice may currently be underestimating the importance of coccygeal pressure in response to pressure-relieving activities. Graphical abstract.

Adaptive Noise Reduction for Sound Event Detection Using Subband-Weighted NMF.

Sound event detection in real-world environments suffers from the interference of non-stationary and time-varying noise. This paper presents an adaptive noise reduction method for sound event detection based on non-negative matrix factorization (NMF). First, a scheme for noise dictionary learning from the input noisy signal is employed by the technique of robust NMF, which supports adaptation to noise variations. The estimated noise dictionary is used to develop a supervised source separation framework in combination with a pre-trained event dictionary. Second, to improve the separation quality, we extend the basic NMF model to a weighted form, with the aim of varying the relative importance of the different components when separating a target sound event from noise. With properly designed weights, the separation process is forced to rely more on those dominant event components, whereas the noise gets greatly suppressed. The proposed method is evaluated on a dataset of the rare sound event detection task of the DCASE 2017 challenge, and achieves comparable results to the top-ranking system based on convolutional recurrent neural networks (CRNNs). The proposed weighted NMF method shows an excellent noise reduction ability, and achieves an improvement of an F-score by 5%, compared to the unweighted approach.

Bilevel Optimization Using Stationary Point of Lower-Level Objective Function for Discriminative Basis Learning in Nonnegative Matrix Factorization

In this letter, we address an audio signal separation problem and propose a new effective algorithm for solving a bilevel optimization in discriminative nonnegative matrix factorization (NMF). Recently, discriminative training of NMF bases has been developed for better signal separation in supervised NMF (SNMF), which exploits a priori training of given sample signals. The optimization in this method consists of a simultaneous minimization of two objective functions, resulting in a bilevel optimization problem with SNMF (BiSNMF), where conventional methods approximately solve this optimization. To strictly solve BiSNMF, we introduce a new algorithm with the following two features: (a) conversion of the optimization constraint into a penalty term and (b) optimization of the reformulated problem on the basis of a multiplicative steepest descent, ensuring the nonnegativity of variables. Experiments on music signal separation show the efficacy of the proposed algorithm.

Training and compensation of class-conditioned NMF bases for speech enhancement

Abstract In this paper, we introduce a training and compensation algorithm of the class-conditioned basis vectors in the non-negative matrix factorization (NMF) model for single-channel speech enhancement. The main goal is to estimate the basis vectors of different signal sources in a way that prevents them from representing each other, in order to reduce the residual noise components that have features similar to the speech signal. During the proposed training stage, the basis matrices for the clean speech and noises are estimated jointly by constraining them to belong to different classes. To this end, we employ the probabilistic generative model (PGM) of classification, specified by class-conditional densities, as an a priori distribution for the basis vectors. The update rules of the NMF and the PGM parameters of classification are jointly obtained by using the variational Bayesian expectation-maximization (VBEM) algorithm, which guarantees convergence to a stationary point. Another goal of the proposed algorithm is to handle a mismatch between the characteristics of the training and test data. This is accomplished during the proposed enhancement stage, where we implement a basis compensation scheme. Specifically, we use extra free basis vectors to capture the features that are not included in the training data. Objective experimental results for different combination of speaker and noise types show that the proposed algorithm can provide better speech enhancement performance than the benchmark algorithms under various conditions.

A Novel Regularized Nonnegative Matrix Factorization for Spectral-Spatial Dimension Reduction of Hyperspectral Imagery

Dimension reduction (DR) is an essential preprocessing for hyperspectral image (HSI) classification. Recently, nonnegative matrix factorization (NMF) has been shown as an effective tool for the DR of hyperspectral data given the fact that it provides interpretable results. However, the basic NMF ignores the geometric structure information of the HSI data, thus limiting its performance. To this end, a novel regularized NMF method, termed NMF with adaptive graph regularizer (NMFAGR), is proposed for the spectral-spatial dimension reduction of hyperspectral data in this paper. Specifically, to enhance the preservation ability of the geometric structure information, the NMFAGR performs the dimension reduction and graph learning simultaneously. Regarding the mutual correlation between these two tasks, a graph regularizer is added as an interaction. Moreover, to effectively utilize complementary information among spectral-spatial features, the NMFAGR allocates feature weight factors automatically without requiring any additional parameters. An efficient algorithm is utilized to solve the optimization problem. The effectiveness of the proposed method is demonstrated on three benchmark hyperspectral data sets through experimentation.

Incremental basis estimation adopting global k-means algorithm for NMF-based noise reduction

Nonnegative matrix factorization (NMF) is a data decomposition technique enabling to discover meaningful latent nonnegative components. Since, however, the objective function of NMF is non-convex, the performance of the source separation can degrade when the iterative update of the basis matrix in the training procedure is stuck to a poor local minimum. In most of the previous studies, the whole basis matrix for a specific source is iteratively updated to minimize a certain objective function with random initialization although a few approaches have been proposed for the systematic initialization of the basis matrix such as the singular value decomposition and k-means clustering. In this paper, we propose an approach to robust bases estimation in which an incremental strategy is adopted. Based on an analogy between clustering and NMF analysis, we estimate the NMF bases in a similar way to the global k-means algorithm popular in the data clustering area. Experiments on audio separation from noise showed that the proposed methods outperformed the conventional NMF technique using random initialization by about 2.04dB and 2.34dB in signal-to-distortion ratio when the target source was speech and violin, respectively.

A non-negative matrix factorization model based on the zero-inflated Tweedie distribution

Non-negative matrix factorization (NMF) is a technique of multivariate analysis used to approximate a given matrix containing non-negative data using two non-negative factor matrices that has been applied to a number of fields. However, when a matrix containing non-negative data has many zeroes, NMF encounters an approximation difficulty. This zero-inflated situation occurs often when a data matrix is given as count data, and becomes more challenging with matrices of increasing size. To solve this problem, we propose a new NMF model for zero-inflated non-negative matrices. Our model is based on the zero-inflated Tweedie distribution. The Tweedie distribution is a generalization of the normal, the Poisson, and the gamma distributions, and differs from each of the other distributions in the degree of robustness of its estimated parameters. In this paper, we show through numerical examples that the proposed model is superior to the basic NMF model in terms of approximation of zero-inflated data. Furthermore, we show the differences between the estimated basis vectors found using the basic and the proposed NMF models for $$\beta $$β divergence by applying it to real purchasing data.

Non-Negative Matrix Factorization Based UKF Algorithm for Constant Modulus Signals in Adaptive Beamforming

Blind adaptive beamforming is getting appreciated for its various applications in contemporary communication systems where sources are statistically dependent or independent that are allowed to formulate new algorithms. Qualitative performance and time complexity are the main issues. In this paper, we propose a technique for constant modulus signals applying basic non-negative matrix factorization (BNMF) in blind adaptive beamforming environment. We compared the existing Unscented Kalman Filter based Constant Modulus Algorithm (UKF-CMA) with proposed NMF-UKF-CMA algorithm. We see there is a better improvement of sensor array gain, signal to interference plus noise ratio (SINR) and mean squared deviation (MSD) as the noise variance and the array size increase with reduced computational complexity with the UKF-CMA.

Categorical, low-dimensional decomposition of human odor space with non-negative matrix factorization

Recent studies using Principal Components Analysis (PCA) support low-dimensional models of odor space, in which one or two dimensions - with hedonic valence featuring prominently - explain most odor variability. Here we use non-negative matrix factorization (NMF) - a nonlinear optimization method - to discover an alternative, reduced-dimensional representation of the Dravnieks odor database (144 odors x 146 descriptors). NMF is theoretically well-suited for these types of analyses, as odor profiling data is inherently non-negative (e.g. descriptors either apply, or do not). We divided the dataset into training and testing halves, and found that RMSD testing error attained a minimum for subspace choice of 25, motivating this as an upper bound for odor perceptual space dimensionality. More parsimonious representations were found by comparing reconstruction errors (fraction of unexplained variance) of NMF with reconstruction errors of PCA on scrambled data (PCAsd). For subspace sizes > 10, NMF error was indistinguishable from PCAsd error, indicating no gain in retaining more than 10 perceptual dimensions. As is typical of NMF basis sets, the 10 odor dimensions we obtain are sparse (only a small subset of the 146 descriptors apply), and categorical (represent a positive valued quality). Moreover, these 10 dimensions were near-orthogonal, with a mean angle of 73 degrees between all pairs of basis vectors. Investigating the distribution of odors in this 10-dimensional space, we find marked clustering (Figure 1), with each odor being well-defined by its membership in a single dimension, and to the exclusion of others. In ongoing work, we are using graph-kernel methods to define a rudimentary mapping between physicochemical features of odorants and the 10 descriptor dimensions.

Evaluation of Distance Measures For NMF-Based Face Image Applications

Non-negative matrix factorization (NMF) is an increasingly popular feature extraction method. Since it is designed to fit training samples using linear combination of non-negative basis vectors, it is particular suitable for imageapplications as it affords intuitive localized interpretations.However, in this space defined by the NMF basis images,there has not been any systematic research to identifysuitable distance measure for NMF-based data classification. In this paper, the performance of 19 distance measuresbetween feature vectors is evaluated based on the result ofthe NMF algorithm for face recognition, which include most of the standard distance measures used in face recognition,as well as two new non-negative vector similarity coefficient-based (NVSC) distances that we recommend for use in NMF-based pattern recognition. Recognition experiments are performed using the CMU AMP Face Expression database, CBCL2 database, MIT-CBCL database, YaleB database, and FERET database. We also compared the performance of NMF with Eigenface method and showed that the NMF algorithm using the NVSC distance yielded the best recognition results.

Nonnegative Matrix Factorization: A Comprehensive Review

Nonnegative Matrix Factorization (NMF), a relatively novel paradigm for dimensionality reduction, has been in the ascendant since its inception. It incorporates the nonnegativity constraint and thus obtains the parts-based representation as well as enhancing the interpretability of the issue correspondingly. This survey paper mainly focuses on the theoretical research into NMF over the last 5 years, where the principles, basic models, properties, and algorithms of NMF along with its various modifications, extensions, and generalizations are summarized systematically. The existing NMF algorithms are divided into four categories: Basic NMF (BNMF), Constrained NMF (CNMF), Structured NMF (SNMF), and Generalized NMF (GNMF), upon which the design principles, characteristics, problems, relationships, and evolution of these algorithms are presented and analyzed comprehensively. Some related work not on NMF that NMF should learn from or has connections with is involved too. Moreover, some open issues remained to be solved are discussed. Several relevant application areas of NMF are also briefly described. This survey aims to construct an integrated, state-of-the-art framework for NMF concept, from which the follow-up research may benefit.

Multichannel Extensions of Non-Negative Matrix Factorization With Complex-Valued Data

This paper presents new formulations and algorithms for multichannel extensions of non-negative matrix factorization (NMF). The formulations employ Hermitian positive semidefinite matrices to represent a multichannel version of non-negative elements. Multichannel Euclidean distance and multichannel Itakura-Saito (IS) divergence are defined based on appropriate statistical models utilizing multivariate complex Gaussian distributions. To minimize this distance/divergence, efficient optimization algorithms in the form of multiplicative updates are derived by using properly designed auxiliary functions. Two methods are proposed for clustering NMF bases according to the estimated spatial property. Convolutive blind source separation (BSS) is performed by the multichannel extensions of NMF with the clustering mechanism. Experimental results show that 1) the derived multiplicative update rules exhibited good convergence behavior, and 2) BSS tasks for several music sources with two microphones and three instrumental parts were evaluated successfully.