Alternating Nonnegative Least Squares Research Articles

A block column iteration for nonnegative matrix factorization

Alternating nonnegative least squares (ANLS) framework is a popular method for nonnegative matrix factorization. This approach, at each step, substitutes the non-convex NMF problem with two nonnegativities constrained least-squares subproblems, so the method for solving these subproblems is critical. This paper adopts a version of block-column iterative methods with nonnegativity constraints (BCI-NC) for solving the subproblems and presents an efficient algorithm in NMF decomposition based on the ANLS framework. The advantages of this method include simplicity and ease of implementation. Also, BCI-NC determines the step lengths without time-consuming line searches, unlike the existing ones. Numerical experiments on real-world image and text datasets show that the proposed algorithm is efficient. We observe that BCI-NC provides the best performance in balancing between accuracy and computation time.

Bayesian Subpixel Mapping of Hyperspectral Imagery via Discrete Endmember Variability Mixture Model and Markov Random Field

Although Bayesian methods have been very effective for spatial-spectral analysis of hyperspectral imagery (HSI), they had not been fully explored for enhanced sub-pixel mapping (SPM) by simultaneously considering several key issues i.e., endmember variability, the discrete nature of subpixel class labels and the spatial information in HSI. Therefore, we propose a new Bayesian SPM method based on discrete endmember variability mixture model (DEMM) and Markov random field (MRF), which has three main characteristics. First, DEMM allows the advanced SPM by completely accounting for the endmember-abundance patterns of each pixel to accommodate the endmember variability, the discrete hidden class label field of subpixels while taking into account the noise heterogeneity effect. Second, the discrete class label field modeled by MRF together with the DEMM, which can be integrated into a novel Bayesian model to better exploit the spatial contextual and spectral information. Third, the resulting Bayesian model can be efficiently solved by a designed expectation-maximization (EM) iteration, where E-step estimates the subpixel class label field using a simulated annealing (SA) algorithm and M-step estimates the endmembers for each pixel in HSI using alternating nonnegative least squares (ANLS) approach. The experimental results on three HSI datasets demonstrate that the proposed approach outperforms previously available SPM techniques.

Adaptive computation of the Symmetric Nonnegative Matrix Factorization (SymNMF)

Symmetric Nonnegative Matrix Factorization (SymNMF) provides a symmetric nonnegative low rank decomposition of symmetric matrices. It has found application in several domains such as data mining, bioinformatics and graph clustering. In this paper SymNMF is obtained by solving a penalized nonsymmetric minimization problem. Instead of letting the penalizing parameter increase according to an a priori fixed rule, as suggested in literature, we propose a heuristic approach based on an adaptive technique. The factorization is performed by applying, as inner-outer procedure, the Alternating Nonnegative Least Squares (ANLS). The inner problems are solved by two nonnegative quadratic optimization methods: an Active-Set-like method (BPP) and the Greedy Coordinate Descent method (GCD). Extensive experimentation shows that the proposed heuristic approach is effective with both the inner methods, GCD outperforming BPP in terms of computational time complexity.

Spatiotemporal Organization of Biofilm Matrix Revealed by Confocal Raman Mapping Integrated with Non-negative Matrix Factorization Analysis.

Biofilms are microbial aggregates of microorganisms surrounded by a hydrogel-like matrix formed by extracellular polymeric substances (EPS). The formation of biofilms is intrinsically complex, from the attachment of microbial cells to the dispersion of the biofilm. Meanwhile, the three-dimensional framework built up by EPS changes with time and protects the microorganisms against environmental stress. Simultaneously acquiring chemical and structural information within the biofilm matrix is vital for the cognition and regulation of biofilms, yet it remains a great challenge due to the sample complexity and the limited approaches. In this study, confocal Raman microscopy and non-negative matrix factorization (NMF) analysis were combined to investigate spatiotemporal organization of Escherichia coli biofilms during development at molecular-level detail. The alternating non-negative least-squares (ANLS) approach was incorporated with the sequential coordinate-wise descent (SCD) algorithm to realize the NMF analysis for the large-scale hyperspectral data set. As a result, three components, including bacteria, protein, and polyhydroxybutyrate (PHB), were successfully resolved from the spectra of E. coli biofilm. Furthermore, the structural changes of biofilms could be visualized and quantified by their abundances derived from the NMF analysis, which might be related to the nutrient and oxygen gradient and physiological functions. This methodology provides a comprehensive understanding of the chemical constituents and their spatiotemporal distribution within the biofilm matrix. Furthermore, it also shows great potential for the analysis of unknown and complex biological samples with 3D Raman mapping.

A hybrid algorithm for low-rank approximation of nonnegative matrix factorization

Nonnegative matrix factorization (NMF) is a recently developed method for data analysis. So far, most of known algorithms for NMF are based on alternating nonnegative least squares (ANLS) minimization of the squared Euclidean distance between the original data matrix and its low-rank approximation. In this paper, we first develop a new NMF algorithm, in which a Procrustes rotation and a nonnegative projection are alternately performed. The new algorithm converges very rapidly. Then, we propose a hybrid NMF (HNMF) algorithm that combines the new algorithm with the low-rank approximation based NMF (lraNMF) algorithm. Furthermore, we extend the HNMF algorithm to nonnegative Tucker decomposition (NTD), which leads to a hybrid NTD (HNTD) algorithm. The simulations verify that the HNMF algorithm performs well under various noise conditions, and HNTD has a comparable performance to the low-rank approximation based sequential NTD (lraSNTD) algorithm for sparse representation of tensor objects.

An Inexact Update Method with Double Parameters for Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) has been used as a powerful date representation tool in real world, because the nonnegativity of matrices is usually required. In recent years, many new methods are available to solve NMF in addition to multiplicative update algorithm, such as gradient descent algorithms, the active set method, and alternating nonnegative least squares (ANLS). In this paper, we propose an inexact update method, with two parameters, which can ensure that the objective function is always descent before the optimal solution is found. Experiment results show that the proposed method is effective.

A proximal ANLS algorithm for nonnegative tensor factorization with a periodic enhanced line search

The Alternating Nonnegative Least Squares (ANLS) method is commonly used for solving nonnegative tensor factorization problems. In this paper, we focus on algorithmic improvement of this method. We present a Proximal ANLS (PANLS) algorithm to enforce convergence. To speed up the PANLS method, we propose to combine it with a periodic enhanced line search strategy. The resulting algorithm, PANLS/PELS, converges to a critical point of the nonnegative tensor factorization problem under mild conditions. We also provide some numerical results comparing the ANLS and PANLS/PELS methods.

Solving non-negative matrix factorization by alternating least squares with a modified strategy

Non-negative matrix factorization (NMF) is a method to obtain a representation of data using non-negativity constraints. A popular approach is alternating non-negative least squares (ANLS). As is well known, if the sequence generated by ANLS has at least one limit point, then the limit point is a stationary point of NMF. However, no evdience has shown that the sequence generated by ANLS has at least one limit point. In order to overcome this shortcoming, we propose a modified strategy for ANLS in this paper. The modified strategy can ensure the sequence generated by ANLS has at least one limit point, and this limit point is a stationary point of NMF. The results of numerical experiments are reported to show the effectiveness of the proposed algorithm.

Fast Nonnegative Matrix Factorization: An Active-Set-Like Method and Comparisons

Nonnegative matrix factorization (NMF) is a dimension reduction method that has been widely used for numerous applications, including text mining, computer vision, pattern discovery, and bioinformatics. A mathematical formulation for NMF appears as a nonconvex optimization problem, and various types of algorithms have been devised to solve the problem. The alternating nonnegative least squares (ANLS) framework is a block coordinate descent approach for solving NMF, which was recently shown to be theoretically sound and empirically efficient. In this paper, we present a novel algorithm for NMF based on the ANLS framework. Our new algorithm builds upon the block principal pivoting method for the nonnegativity-constrained least squares problem that overcomes a limitation of the active set method. We introduce ideas that efficiently extend the block principal pivoting method within the context of NMF computation. Our algorithm inherits the convergence property of the ANLS framework and can easily be extended to other constrained NMF formulations. Extensive computational comparisons using data sets that are from real life applications as well as those artificially generated show that the proposed algorithm provides state-of-the-art performance in terms of computational speed.