Non-negative Basis Research Articles

An information-theoretic approach to basis-set fitting of electron densities and other non-negative functions.

The numerical ill-conditioning associated with approximating an electron density with a convex sum of Gaussian or Slater-type functions is overcome by using the (extended) Kullback-Leibler divergence to measure the deviation between the target and approximate density. The optimized densities are non-negative and normalized, and they are accurate enough to be used in applications related to molecular similarity, the topology of the electron density, and numerical molecular integration. This robust, efficient, and general approach can be used to fit any non-negative normalized functions (e.g., the kinetic energy density and molecular electron density) to a convex sum of non-negative basis functions. We present a fixed-point iteration method for optimizing the Kullback-Leibler divergence and compare it to conventional gradient-based optimization methods. These algorithms are released through the free and open-source BFit package, which also includes a L2-norm squared optimization routine applicable to any square-integrable scalar function.

Initialization for non-negative matrix factorization: a comprehensive review

Non-negative matrix factorization (NMF) has become a popular method for representing meaningful data by extracting a non-negative basis feature from an observed non-negative data matrix. Some of the unique features of this method in identifying hidden data put this method amongst the powerful methods in the machine learning area. The NMF is a known non-convex optimization problem and the initial point has a significant effect on finding an efficient local solution. In this paper, we investigate the most popular initialization procedures proposed for NMF so far. We describe each method and present some of their advantages and disadvantages. Finally, some numerical results to illustrate the performance of each algorithm are presented.

A construction of edge B-spline functions for a C1 polynomial spline on two triangles and its application to Argyris type splines

Given two triangles in a planar domain sharing an edge and forming a convex quadrilateral, it is shown how to construct a non-negative basis for C1 splines that restrict to polynomials of a total degree higher than one on each of the triangles. The representation may be seen as a generalization of the Bernstein–Bézier form of a spline on every separate triangle, and the main challenge in its development is the construction of basis functions associated with the common edge. This novel concept is aimed to be used in assembling B-spline-like bases for C1 splines on triangulations, as it is demonstrated for Argyris type splines of degree higher than five on triangulations with flippable edges.

Deep Non-Negative Matrix Factorization Architecture Based on Underlying Basis Images Learning

The non-negative matrix factorization (NMF) algorithm represents the original image as a linear combination of a set of basis images. This image representation method is in line with the idea of "parts constitute a whole" in human thinking. The existing deep NMF performs deep factorization on the coefficient matrix. In these methods, the basis images used to represent the original image is essentially obtained by factorizing the original images once. To extract features reflecting the deep localization characteristics of images, a novel deep NMF architecture based on underlying basis images learning is proposed for the first time. The architecture learns the underlying basis images by deep factorization on the basis images matrix. The deep factorization architecture proposed in this paper has strong interpretability. To implement this architecture, this paper proposes a deep non-negative basis matrix factorization algorithm to obtain the underlying basis images. Then, the objective function is established with an added regularization term, which directly constrains the basis images matrix to obtain the basis images with good local characteristics, and a regularized deep non-negative basis matrix factorization algorithm is proposed. The regularized deep nonlinear non-negative basis matrix factorization algorithm is also proposed to handle pattern recognition tasks with complex data. This paper also theoretically proves the convergence of the algorithm. Finally, the experimental results show that the deep NMF architecture based on the underlying basis images learning proposed in this paper can obtain better recognition performance than the other state-of-the-art methods.

Maximum penalized likelihood estimation of additive hazards models with partly interval censoring

Existing likelihood methods for the additive hazards model with interval censored survival data are limited and often ignore the non-negative constraints on hazards. This paper proposes a maximum penalized likelihood method to fit additive hazards models with interval censoring. Our method firstly models the baseline hazard using a finite number of non-negative basis functions, and then regression coefficients and baseline hazard are estimated simultaneously by maximizing a penalized log-likelihood function, where a penalty function is introduced to regularize the baseline hazard estimate. In the estimation procedure, non-negative constraints are imposed on both the baseline hazard and the hazard of each subject. A primal–dual interior-point algorithm is applied to solve the constrained optimization problem. Asymptotic properties are obtained and a simulation study is conducted for assessment of the proposed method.

Non-negative matrix factorization via discriminative label embedding for pattern classification

As one of the most commonly used dimension reduction approaches, discriminant non-negative matrix factorization (NMF) has been widely used for data representation in the pattern classification task. However, the previous discriminant NMFs emphasize the Fisher criterion or maximum margin criterion which has high requirement to the distribution of data. Therefore, this work proposes a discriminative label embedded NMF (LENMF) algorithm. LENMF takes into account the discriminative label embedding to obtain the low-dimensional projected data and orthogonal property of the non-negative basis to strength the ability of parts-based representation. Besides, LENMF is extended in the kernel space to explore the nonlinear relations of data. By integrating the non-negative constraint, discriminative label embedding, and the orthogonal property into the proposed objective, the multiplicative updating rules have been given in this work. Experiment results on the challenging face, object, document, and digit databases illustrate the performance of the proposed algorithm.

Label and orthogonality regularized non-negative matrix factorization for image classification

As one of the most popular data-representation methods, non-negative matrix factorization (NMF) has been widely used in image processing and pattern recognition. Compared with other dimension reduction methods, we can interpret the data with psychological intuition using NMF since NMF can decompose the whole into visual parts by learning the non-negative basis. However, the original NMF lacks of extracting the discriminant information of the data for the image classification task. For enhancing the discriminant and parts-based interpretability, this work proposes a label and orthogonality regularized NMF (LONMF) algorithm based on the squared Euclidean distance. LONMF takes into account the label consistence with the low-dimensional projected data and orthogonal property of the non-negative basis. By integrating the non-negative constraint, label consistence, and orthogonal property into the objective function, the efficient updating procedure can obtain a discriminant basis matrix. Meanwhile, we design a linear classifier using the projected data to guide the label for efficient image classification task. Experiment results of the competitive NMF variants on the challenging digit and face databases demonstrate the effectiveness of the proposed LONMF algorithm.

Non-negative Matrix Factorization: Robust Extraction of Extended Structures

Abstract We apply the vectorized non-negative matrix factorization (NMF) method to the post-processing of the direct imaging data of exoplanetary systems such as circumstellar disks. NMF is an iterative approach, which first creates a nonorthogonal and non-negative basis of components using the given reference images and then models a target with the components. The constructed model is then rescaled with a factor to compensate for the contribution from the disks. We compare NMF with existing methods (classical reference differential imaging method, and the Karhunen–Loève image projection algorithm) using synthetic circumstellar disks and demonstrate the superiority of NMF: with no need of prior selection of references, NMF not only can detect fainter circumstellar disks but also better preserves their morphology and does not require forward modeling. As an application to a well-known disk example, we process the archival Hubble Space Telescope STIS coronagraphic observations of HD 181327 with different methods and compare them, and NMF is able to extract some circumstellar materials inside the primary ring for the first time. In an appendix, we mathematically investigate the stability of NMF components during the iteration and the linearity of NMF modeling.

Joint Linear Regression and Nonnegative Matrix Factorization Based on Self-Organized Graph for Image Clustering and Classification

Nonnegative matrix factorization (NMF) technique has been developed successfully to represent the intuitively meaningful feature of data. A suitable representation can faithfully preserve the intrinsic structure of data. Due to the fact that it introduces the label information, semi-supervised NMF has been demonstrated more advantageous in image representation than original NMF. However, previous semi-supervised NMF variants construct a label indicator matrix only for tagging the labeled data and not being optimized together with the matrix factorization. It is short of label propagation and fails to work for predicting the attribution of data. Moreover, the transductive semi-supervised NMF variants cannot dispose the prediction of unseen data, restricting the application of NMF. In this paper, a joint optimization framework of linear regression and NMF (LR-NMF) based on the self-organized graph is proposed for a completed task which simultaneously takes into account image representation and attribution prediction. By minimizing the proposed objective, three interactive threads are running: decomposing the data into nonnegative basis matrix and the corresponding representation, linear regression using the nonnegative representation, and label propagation based on the self-organized graph which is defined in the feature space. The products of LR-NMF can be viewed as extracting nonnegative feature for clustering, meanwhile, they can be used to solve the out-of-sample problem for classification. Extensive clustering and classification experiments on the digit, face, and object challenging data sets are presented to show the efficacy of the proposed LR-NMF algorithm.

A multi-view-group non-negative matrix factorization approach for automatic image annotation

In automatic image annotation (AIA) different features describe images from different aspects or views. Part of information embedded in some views is common for all views, while other parts are individual and specific. In this paper, we present the Mvg-NMF approach, a multi-view-group non-negative matrix factorization (NMF) method for an AIA system which considers both common and individual factors. The NMF framework discovers a latent space by decomposing data into a set of non-negative basis vectors and coefficients. The views divided into homogeneous groups and latent spaces are extracted for each group. After mapping the test images into these spaces, a unified distance matrix is computed from the distance between images in all spaces. Then a search-based method is used to propagate tags from the nearest neighbors to test images. The evaluation on three datasets commonly used for image annotation showed that the Mvg-NMF is highly competitive with the recent state-of-the-art works.

Reduced integration at superconvergent points in isogeometric analysis

We propose a new reduced integration rule for isogeometric analysis (IGA) based on the concept of variational collocation. It has been recently shown that, if a discrete space is constructed by smooth and pointwise non-negative basis functions, there exists a set of points – named Cauchy–Galerkin (CG) points – such that collocation performed at these points can reproduce the Galerkin solution of various boundary value problems exactly. Since CG points are not known a-priori, estimates are necessary in practice and can be found based on superconvergence theory. In this contribution, we explore the use of estimated CG points (i.e. superconvergent points) as numerical quadrature points to obtain an efficient and stable reduced quadrature rule in IGA. We use the weighted residual formulation as basis for our new quadrature rule, so that the proposed approach can be considered intermediate between the standard (accurately integrated) Galerkin variational formulation and the direct evaluation of the strong form in collocation approaches. The performance of the method is demonstrated by several examples. For odd degrees of discretization, we obtain spatial convergence rates and accuracy very close to those of accurately integrated standard Galerkin with a quadrature rule of two points per parametric direction independently of the degree.

Evolutionary nonnegative matrix factorization with adaptive control of cluster quality

Nonnegative matrix factorization (NMF) approximates a given data matrix using linear combinations of a small number of nonnegative basis vectors, weighted by nonnegative encoding coefficients. This enables the exploration of the cluster structure of the data through the examination of the values of the encoding coefficients and therefore, NMF is often used as a popular tool for clustering analysis. However, its encoding coefficients do not always reveal a satisfactory cluster structure. To improve its effectiveness, a novel evolutionary strategy is proposed here to drive the iterative updating scheme of NMF and generate encoding coefficients of higher quality that are capable of offering more accurate and sharper cluster structures. The proposed hybridization procedure that relies on multiple initializations reinforces the robustness of the solution. Additionally, three evolving rules are designed to simultaneously boost the cluster quality and the reconstruction error during the iterative updates. Any clustering performance measure, such as either an internal one relying on the data itself or an external based on the availability of ground truth information, can be employed to drive the evolving procedure. The effectiveness of the proposed method is demonstrated via careful experimental designs and thorough comparative analyses using multiple benchmark datasets.

Image annotation using multi-view non-negative matrix factorization with different number of basis vectors

Automatic Image Annotation (AIA) helps image retrieval systems by predicting tags for images. In this paper, we propose an AIA system using Non-negative Matrix Factorization (NMF) framework. The NMF framework discovers a latent space, by factorizing data into a set of non-negative basis and coefficients. To model the images, multiple features are extracted, each one represents images from a specific view. We use multi-view graph regularization NMF and allow NMF to choose a different number of basis vectors for each view. For tag prediction, each test image is mapped onto the multiple latent spaces. The distances of images in these spaces are used to form a unified distance matrix. The weights of distances are learned automatically. Then a search-based method is used to predict tags based on tags of nearest neighbors’. We evaluate our method on three datasets and show that it is competitive with the current state-of-the-art methods.

Semi-supervised Nonnegative Matrix Factorization with Commonness Extraction

Standard nonnegative matrix factorization extracts nonnegative bases for nonnegative representation, which, however, considers only features, not commonness. In addition, the standard NMF is an unsupervised learning method that cannot fully utilize label information if it exists. In this paper, we present a semi-supervised commonness NMF technique that incorporates samples' commonness and label information into the optimization model. Naturally, the commonness vector should be constrained by nonnegativity and will degenerate to zero if no commonness exists. We develop a multiplicative update rule to solve the model, which has properties comparable to those of the standard NMF with automatic satisfaction of the nonnegativity constraints, monotonicity without the need for any adjustable learning rate and a low computational overhead. Through experiments on the standard databases, we analyze the behavior of the proposed method, which exhibits a performance that is favorably superior with respect to commonness extraction and clustering accuracy.

Sequential dimensionality reduction for extracting localized features

Linear dimensionality reduction techniques are powerful tools for image analysis as they allow the identification of important features in a data set. In particular, nonnegative matrix factorization (NMF) has become very popular as it is able to extract sparse, localized and easily interpretable features by imposing an additive combination of nonnegative basis elements. Nonnegative matrix underapproximation (NMU) is a closely related technique that has the advantage to identify features sequentially. In this paper, we propose a variant of NMU that is particularly well suited for image analysis as it incorporates the spatial information, that is, it takes into account the fact that neighboring pixels are more likely to be contained in the same features, and favors the extraction of localized features by looking for sparse basis elements. We show that our new approach competes favorably with comparable state-of-the-art techniques on synthetic, facial and hyperspectral image data sets.

The variational collocation method

We propose the variational collocation method for the numerical solution of partial differential equations. The conceptual basis is the establishment of a direct connection between the Galerkin method and the classical collocation methods, with the perspective of achieving the accuracy of the former with a computational cost of one point evaluation per degree of freedom as in the latter. Variational collocation requires a discrete space constructed by smooth and pointwise non-negative basis functions, which makes the approach immediately applicable to isogeometric analysis and some meshfree methods. In this paper, we concentrate on isogeometric analysis and demonstrate that there exists a set of points such that collocation of the strong form at these points produces the Galerkin solution exactly. We provide an estimate of these points and show that applying isogeometric collocation at the estimated points completely solves the well-known odd/even discrepancy in the order of spatial convergence. We demonstrate the potential of variational collocation with examples of linear and non-linear elasticity as well as Kirchhoff plates.

Online Convolutive Non-Negative Bases Learning for Speech Enhancement

An online version of convolutive non-negative sparse coding (CNSC) with the generalized Kullback-Leibler (K-L) divergence is proposed to adaptively learn spectral-temporal bases from speech streams. The proposed scheme processes training data piece-by-piece and incrementally updates learned bases with accumulated statistics to overcome the inefficiency of its offline counterpart in processing large scale or streaming data. Compared to conventional non-negative sparse coding, we utilize the convolutive model within bases, so that each basis is capable of describing a relatively long temporal span of signals, which helps to improve the representation power of the model. Moreover, by incorporating a voice activity detector (VAD), we propose an unsupervised enhancement algorithm that updates the noise dictionary adaptively from non-speech intervals. Meanwhile, for the speech intervals, one can adaptively learn the speech bases by keeping the noise ones fixed. Experimental results show that the proposed algorithm outperforms the competing algorithms substantially, especially when the background noise is highly non-stationary.

Trans-dimensional Bayesian inference for gravitational lens substructures

We introduce a Bayesian solution to the problem of inferring the density profile of strong gravitational lenses when the lens galaxy may contain multiple dark or faint substructures. The source and lens models are based on a superposition of an unknown number of non-negative basis functions (or blobs) whose form was chosen with speed as a primary criterion. The prior distribution for the blobs' properties is specified hierarchically, so the mass function of substructures is a natural output of the method. We use reversible jump Markov Chain Monte Carlo (MCMC) within Diffusive Nested Sampling (DNS) to sample the posterior distribution and evaluate the marginal likelihood of the model, including the summation over the unknown number of blobs in the source and the lens. We demonstrate the method on two simulated data sets: one with a single substructure, and one with ten. We also apply the method to the g-band image of the Cosmic Horseshoe system, and find evidence for more than zero substructures. However, these have large spatial extent and probably only point to misspecifications in the model (such as the shape of the smooth lens component or the point spread function), which are difficult to guard against in full generality.

Decomposing time series data by a non-negative matrix factorization algorithm with temporally constrained coefficients.

The non-negative matrix factorization algorithm (NMF) decomposes a data matrix into a set of non-negative basis vectors, each scaled by a coefficient. In its original formulation, the NMF assumes the data samples and dimensions to be independently distributed, making it a less-than-ideal algorithm for the analysis of time series data with temporal correlations. Here, we seek to derive an NMF that accounts for temporal dependencies in the data by explicitly incorporating a very simple temporal constraint for the coefficients into the NMF update rules. We applied the modified algorithm to 2 multi-dimensional electromyographic data sets collected from the human upper-limb to identify muscle synergies. We found that because it reduced the number of free parameters in the model, our modified NMF made it possible to use the Akaike Information Criterion to objectively identify a model order (i.e., the number of muscle synergies composing the data) that is more functionally interpretable, and closer to the numbers previously determined using ad hoc measures.

Formula omitted] cubic splines on Powell–Sabin triangulations

A bivariate C1 cubic spline space on a triangulation with Powell–Sabin refinement which extends the well-known C1 quadratic spline space and has a nested structure is introduced. A construction of a locally supported basis that forms a partition of unity is presented based on choosing particular triangles and line segments in the domain. Further, it is shown how these objects can be determined in order to obtain nonnegative basis functions under a natural restriction on the Powell–Sabin refinement. Geometrically intuitive B-spline representation is proposed which makes these splines a useful tool for CAGD applications.