Nonnegative Matrix Factorization Initialization Research Articles

Initialization for non-negative matrix factorization: a comprehensive review

Non-negative matrix factorization (NMF) has become a popular method for representing meaningful data by extracting a non-negative basis feature from an observed non-negative data matrix. Some of the unique features of this method in identifying hidden data put this method amongst the powerful methods in the machine learning area. The NMF is a known non-convex optimization problem and the initial point has a significant effect on finding an efficient local solution. In this paper, we investigate the most popular initialization procedures proposed for NMF so far. We describe each method and present some of their advantages and disadvantages. Finally, some numerical results to illustrate the performance of each algorithm are presented.

A Novel Method for Hyperspectral Mineral Mapping Based on Clustering-Matching and Nonnegative Matrix Factorization

The emergence of hyperspectral imagery paved a new way for rapid mineral mapping. As a classical hyperspectral classification method, spectral matching (SM) can automatically map the spatial distribution of minerals without the need for selecting training samples. However, due to the influence of noise, the mapping accuracy of SM is usually poor, and its per-pixel matching method is inefficient to some extent. To solve these problems, we propose an unsupervised clustering-matching mapping method, using a combination of k-means and SM (KSM). First, nonnegative matrix factorization (NMF) is used and combined with a simple and effective NMF initialization method (SMNMF) for feature extraction. Then, k-means is implemented to get the cluster centers of the extracted features and band depth, which are used for clustering and matching, respectively. Finally, dimensionless matching methods, including spectral angle mapper (SAM), spectral correlation angle (SCA), spectral gradient angle (SGA), and a combined matching method (SCGA) are used to match the cluster centers of band depth with a spectral library to obtain the mineral mapping results. A case study on the airborne hyperspectral image of Cuprite, Nevada, USA, demonstrated that the average overall accuracies of KSM based on SAM, SCA, SGA, and SCGA are approximately 22%, 22%, 35%, and 33% higher than those of SM, respectively, and KSM can save more than 95% of the mapping time. Moreover, the mapping accuracy and efficiency of SMNMF are about 15% and 38% higher than those of the widely used NMF initialization method. In addition, the proposed SCGA could achieve promising mapping results at both high and low signal-to-noise ratios compared with other matching methods. The mapping method proposed in this study provides a new solution for the rapid and autonomous identification of minerals and other fine objects.

A Review on Initialization Methods for Nonnegative Matrix Factorization: Towards Omics Data Experiments

Nonnegative Matrix Factorization (NMF) has acquired a relevant role in the panorama of knowledge extraction, thanks to the peculiarity that non-negativity applies to both bases and weights, which allows meaningful interpretations and is consistent with the natural human part-based learning process. Nevertheless, most NMF algorithms are iterative, so initialization methods affect convergence behaviour, the quality of the final solution, and NMF performance in terms of the residual of the cost function. Studies on the impact of NMF initialization techniques have been conducted for text or image datasets, but very few considerations can be found in the literature when biological datasets are studied, even though NMFs have largely demonstrated their usefulness in better understanding biological mechanisms with omic datasets. This paper aims to present the state-of-the-art on NMF initialization schemes along with some initial considerations on the impact of initialization methods when microarrays (a simple instance of omic data) are evaluated with NMF mechanisms. Using a series of measures to qualitatively examine the biological information extracted by a given NMF scheme, it preliminary appears that some information (e.g., represented by genes) can be extracted regardless of the initialization scheme used.

Nonnegative tensor decomposition for urban mobility analysis and applications with mobile phone data

This paper reveals the spatial-temporal patterns of urban mobility by exploring massive mobile phone data based on the nonnegative tensor decomposition method. First, human mobility data with the trip origin, destination, and timestamp are formulated to a three-way tensor. Second, the nonnegative Tucker decomposition model is used to reconstruct the core tensor and the factor matrix to extract hidden structures. Third, the model is efficiently estimated using the hierarchical alternating least square nonnegative tensor decomposition (NTD) algorithm with the nonnegative matrix factorization (NMF) initialization. Using the one-week data of over 4 million cell phone users in Hangzhou, China, we evaluate the performance of the proposed method and explore how different initialization strategies affect tensor decomposition performance. The results show that the NMF initialization strategy can speed up the convergence process and achieve a better fit and more stable results than random initialization in tensor decomposition.

Improved SVD-based initialization for nonnegative matrix factorization using low-rank correction

Due to the iterative nature of the most nonnegative matrix factorization (NMF) algorithms, initialization is a key aspect as it significantly influences both the convergence and the final solution obtained. Many initialization schemes have been proposed for NMF, among which one of the most popular class of methods are based on the singular value decomposition (SVD) and clustering. However, these SVD-based initializations as well as clustering based initializations (if they dense their right factor H), do not satisfy a rather natural condition, namely that the error should decrease as the rank of factorization increases. In this paper, we propose a novel SVD-based NMF initialization to specifically address this shortcoming by taking into account the SVD factors that were discarded to obtain a nonnegative initialization. This method, referred to as nonnegative SVD with low-rank correction (NNSVD-LRC), allows us to significantly reduce the initial error at a negligible additional computational cost using the low-rank structure of the discarded SVD factors. NNSVD-LRC has two other advantages compared to other NMF initializations: (1) it provably generates sparse initial factors, and (2) it is faster as it only requires to compute a truncated SVD of rank ⌊r2+1⌋ where r is the factorization rank of the sought NMF decomposition (as opposed to a rank-r truncated SVD for other methods). We show on several standard dense and sparse data sets that our new method competes favorably with state-of-the-art SVD-based and clustering based initializations for NMF.

Ex Vivo Hyperspectral Autofluorescence Imaging and Localization of Fluorophores in Human Eyes with Age-Related Macular Degeneration.

To characterize fluorophore signals from drusen and retinal pigment epithelium (RPE) and their changes in age related macular degeneration (AMD), the authors describe advances in ex vivo hyperspectral autofluorescence (AF) imaging of human eye tissue. Ten RPE flatmounts from eyes with AMD and 10 from eyes without AMD underwent 40× hyperspectral AF microscopic imaging. The number of excitation wavelengths tested was initially two (436 nm and 480 nm), then increased to three (436 nm, 480 nm, and 505 nm). Emission spectra were collected at 10 nm intervals from 420 nm to 720 nm. Non-negative matrix factorization (NMF) algorithms decomposed the hyperspectral images into individual emission spectra and their spatial abundances. These include three distinguishable spectra for RPE fluorophores (S1, S2, and S3) in both AMD and non-AMD eyes, a spectrum for drusen (SDr) only in AMD eyes, and a Bruch’s membrane spectrum that was detectable in normal eyes. Simultaneous analysis of datacubes excited atthree excitation wavelengths revealed more detailed spatial localization of the RPE spectra and SDr within drusen than exciting only at two wavelengths. Within AMD and non-AMD groups, two different NMF initialization methods were tested on each group and converged to qualitatively similar spectra. In AMD, the peaks of the SDr at ~510 nm (436 nm excitation) were particularly consistent. Between AMD and non-AMD groups, corresponding spectra in common, S1, S2, and S3, also had similar peak locations and shapes, but with some differences and further characterization warranted.

An Endmember Initialization Scheme for Nonnegative Matrix Factorization and Its Application in Hyperspectral Unmixing

Nonnegative matrix factorization (NMF) is a blind source separation (BSS) method often used in hyperspectral unmixing. However, it tends to converge to a local optimum. To overcome this limitation, we present a simple, but effective endmember initialization scheme for NMF, which is realized by improving initial values through the application of the automatic target generation process (ATGP) algorithm. The initial spectra and abundances of target endmembers are first obtained using the ATGP algorithm and nonnegative least squares (NNLS) method, respectively. The preliminary results are then optimized through iterative application of NMF. To validate the applicability and effectiveness of the proposed method, we analyzed the improvement of NMF by the ATGP algorithm, using the synthetic hyperspectral data and real hyperspectral images. The results from the proposed method are compared with those of the vertex component analysis (VCA)-NMF algorithm, which uses the VCA algorithm to perform initialization for NMF, the minimum volume constrained NMF (MVC-NMF) algorithm, the traditional two-step VCA-fully-constrained least squares (FCLS) algorithm, which uses the VCA to extract the endmember matrix, and the FCLS algorithm to estimate the abundance matrix. The comparison results prove that proper endmember initialization can help the NMF algorithm yield better estimation results. Through the optimization of target endmembers’ initial values, the proposed ATGP-NMF algorithm can consistently produce good results at a lower computational cost, especially in the case of a real hyperspectral image for which pure pixels do not exist and there is little prior knowledge. With its high applicability and effectiveness, the ATGP-NMF algorithm has a great potential to solve hyperspectral unmixing problems.

Multiple-algorithm parallel fusion of infrared polarization and intensity images based on algorithmic complementarity and synergy

Diverse image fusion methods perform differently. Each method has advantages and disadvantages compared with others. One notion is that the advantages of different image methods can be effectively combined. A multiple-algorithm parallel fusion method based on algorithmic complementarity and synergy is proposed. First, in view of the characteristics of the different algorithms and difference-features among images, an index vector-based feature-similarity is proposed to define the degree of complementarity and synergy. This proposed index vector is a reliable evidence indicator for algorithm selection. Second, the algorithms with a high degree of complementarity and synergy are selected. Then, the different degrees of various features and infrared intensity images are used as the initial weights for the nonnegative matrix factorization (NMF). This avoids randomness of the NMF initialization parameter. Finally, the fused images of different algorithms are integrated using the NMF because of its excellent data fusing performance on independent features. Experimental results demonstrate that the visual effect and objective evaluation index of the fused images obtained using the proposed method are better than those obtained using traditional methods. The proposed method retains all the advantages that individual fusion algorithms have.

Rank-One NMF-Based Initialization for NMF and Relative Error Bounds Under a Geometric Assumption

We propose a geometric assumption on nonnegative data matrices such that under this assumption, we are able to provide upper bounds (both deterministic and probabilistic) on the relative error of nonnegative matrix factorization (NMF). The algorithm we propose first uses the geometric assumption to obtain an exact clustering of the columns of the data matrix; subsequently, it employs several rank-one NMFs to obtain the final decomposition. When applied to data matrices generated from our statistical model, we observe that our proposed algorithm produces factor matrices with comparable relative errors vis-\`a-vis classical NMF algorithms but with much faster speeds. On face image and hyperspectral imaging datasets, we demonstrate that our algorithm provides an excellent initialization for applying other NMF algorithms at a low computational cost. Finally, we show on face and text datasets that the combinations of our algorithm and several classical NMF algorithms outperform other algorithms in terms of clustering performance.

An Optimal Method for the Initialization of Non-negative Matrix Factorization (NMF)

An optimization method of the NMF algorithm initialization is proposed. This optimization method can be easily integrated with the existing initialization methods of NMF algorithm. The strategy is based on the geometric interpretation of NMF, in the convex hull, the intersection point between the connect of two points and the corresponding boundary of probability simplex, is used to update the initial basis vectors corresponding point in the matrix, so that the base vector matrix extended, and it can better contain the original matrix. Many numerical examples show that, compared with the original initialization, this method can obtain better results.

Initialization of nonnegative matrix factorization by Gaussian primaries for reconstruction of spectral data

The Gaussian and Gaussian-sigmoidal primaries are introduced as initialization functions in the nonnegative matrix factorization (NNMF) method to achieve more consistent error statistics. The proposed NNMF method with multiplicative update algorithm is employed for reconstruction of spectral data of Munsell and Colorchecker DC datasets from the corresponding CIEXYZ tristimulus values and the resultant achievements are compared with those obtained from PCA and the classical NNMF techniques. Both colorimetric and spectrophotometric errors between the original and the reconstructed spectra are used for the evaluation of reconstruction methods. While run to run variability of the classical NNMF method with multiplicative update algorithm is significantly remarkable, the stability of the suggested technique is fairly high, similar to the PCA method. The ensuing findings show that the embedding of Gaussian initializing vectors in NNMF method provide the all positive bases that enhance both the accuracy of Gaussian method and the reliability of NNMF technique.

Non‐negative Matrix Factorization for the identification of patterns of atmospheric pressure and geopotential for the Northern Hemisphere

AbstractAs the atmospheric system is characterized by highly complex interactions between a large number of physical variables, it is a challenging task for researchers to break up the complicated structures into a few significant modes of variability.An innovative technique, called Non‐negative Matrix Factorization (NMF), developed to provide a reduced‐dimensional representation of large‐scale non‐negative data and to extract underlying features, is introduced as a tool for meteorological applications. The method is used to decompose space‐time meteorological fields into spatial patterns and associated time indices in order to advance our knowledge of dominant atmospheric processes. For the Northern Hemisphere the 500 hPa geopotential field together with sea level pressure (SLP) are analysed using an up‐to‐date NMF algorithm as well as a helpful method of NMF initialization based on Singular Value Decomposition (SVD). Several NMF patterns correspond to the variations identified by traditional EOF analysis, such as the East Atlantic and the Pacific/North American Pattern. For the North Atlantic Oscillation the NMF identifies sub‐patterns: The positive phase is associated with systems of pressure variation over the Pacific Ocean and the Arctic, respectively, the negative phase resembles the EOF pattern.As the NMF patterns not only represent variability, like EOF patterns, but also are fractions of the total observed values, we demonstrate for a location near Nikolski, Alaska, the temporal development of the geopotential as a superposition of NMF‐factors. Copyright © 2009 Royal Meteorological Society

Clustering-based initialization for non-negative matrix factorization

Non-negative matrix factorization (NMF) is an unsupervised learning algorithm that can extract parts from visual data. The goal of this technique is to find intuitive basis such that training examples can be faithfully reconstructed using linear combination of basis images which are restricted to non-negative values. Thus, NMF basis images can be understood as localized features that correspond better with intuitive notions of parts of images. However, there has been few systematic study to explore various methods for initialization of NMF algorithm, which is crucial for the performance of NMF algorithm in data analysis. In this paper, we discuss a structured NMF initialization scheme based on the clustering method. Comparing with the random initialization in common use, our method achieved faster convergence while maintaining the data structure and also obtained good result for the face recognition task. Furthermore, we also proposed to use a normalized AIC incorporated with our NMF initialization for rank selection of traditional NMF at the cost of much less computational load while obtaining a good performance in face recognition.

Initialization enhancer for non-negative matrix factorization

Non-negative matrix factorization (NMF), proposed recently by Lee and Seung, has been applied to many areas such as dimensionality reduction, image classification image compression, and so on. Based on traditional NMF, researchers have put forward several new algorithms to improve its performance. However, particular emphasis has to be placed on the initialization of NMF because of its local convergence, although it is usually ignored in many documents. In this paper, we explore three initialization methods based on principal component analysis (PCA), fuzzy clustering and Gabor wavelets either for the consideration of computational complexity or the preservation of structure. In addition, the three methods develop an efficient way of selecting the rank of the NMF in low-dimensional space.

Improving non-negative matrix factorizations through structured initialization

In this paper we explore a recent iterative compression technique called non-negative matrix factorization (NMF). Several special properties are obtained as a result of the constrained optimization problem of NMF. For facial images, the additive nature of NMF results in a basis of features, such as eyes, noses, and lips. We explore various methods for efficiently computing NMF, placing particular emphasis on the initialization of current algorithms. We propose using Spherical K-Means clustering to produce a structured initialization for NMF. We demonstrate some of the properties that result from this initialization and develop an efficient way of choosing the rank of the low-dimensional NMF representation.