Non-negative Factor Analysis Research Articles

Adaptive Divergence-Based Non-Negative Latent Factor Analysis of High-Dimensional and Incomplete Matrices From Industrial Applications

Adaptive Divergence-Based Non-Negative Latent Factor Analysis of High-Dimensional and Incomplete Matrices From Industrial Applications

An Alternating-Direction-Method of Multipliers-Incorporated Approach to Symmetric Non-Negative Latent Factor Analysis.

Large-scale undirected weighted networks are frequently encountered in big-data-related applications concerning interactions among a large unique set of entities. Such a network can be described by a Symmetric, High-Dimensional, and Incomplete (SHDI) matrix whose symmetry and incompleteness should be addressed with care. However, existing models fail in either correctly representing its symmetry or efficiently handling its incomplete data. For addressing this critical issue, this study proposes an Alternating-Direction-Method of Multipliers (ADMM)-based Symmetric Non-negative Latent Factor Analysis (ASNL) model. It adopts fourfold ideas: 1) implementing the data density-oriented modeling for efficiently representing an SHDI matrix's incomplete and imbalanced data; 2) separating the non-negative constraints from the decision parameters to avoid truncations during the training process; 3) incorporating the ADMM principle into its learning scheme for fast model convergence; and 4) parallelizing the training process with load balance considerations for high efficiency. Empirical studies on four SHDI matrices demonstrate that ASNL significantly outperforms several state-of-the-art models in both prediction accuracy for missing data of an SHDI and computational efficiency. It is a promising model for handling large-scale undirected networks raised in real applications.

Proximal Alternating-Direction-Method-of-Multipliers-Incorporated Nonnegative Latent Factor Analysis

High-dimensional and incomplete (HDI) data subject to the nonnegativity constraints are commonly encountered in a big data-related application concerning the interactions among numerous nodes. A nonnegative latent factor analysis (NLFA) model can perform representation learning to HDI data efficiently. However, existing NLFA models suffer from either slow convergence rate or representation accuracy loss. To address this issue, this paper proposes a proximal alternating-direction-method-of-multipliers-based nonnegative latent factor analysis (PAN) model with two-fold ideas: 1) adopting the principle of alternating-direction-method-of-multipliers to implement an efficient learning scheme for fast convergence and high computational efficiency; and 2) incorporating the proximal regularization into the learning scheme to suppress the optimization fluctuation for high representation learning accuracy to HDI data. Theoretical studies verify that PAN converges to a Karush-Kuhn-Tucker (KKT) stationary point of its nonnegativity-constrained learning objective with its learning scheme. Experimental results on eight HDI matrices from real applications demonstrate that the proposed PAN model outperforms several state-of-the-art models in both estimation accuracy for missing data of an HDI matrix and computational efficiency.

FISHFactor: a probabilistic factor model for spatial transcriptomics data with subcellular resolution.

Factor analysis is a widely used tool for unsupervised dimensionality reduction of high-throughput datasets in molecular biology, with recently proposed extensions designed specifically for spatial transcriptomics data. However, these methods expect (count) matrices as data input and are therefore not directly applicable to single molecule resolution data, which are in the form of coordinate lists annotated with genes and provide insight into subcellular spatial expression patterns. To address this, we here propose FISHFactor, a probabilistic factor model that combines the benefits of spatial, non-negative factor analysis with a Poisson point process likelihood to explicitly model and account for the nature of single molecule resolution data. In addition, FISHFactor shares information across a potentially large number of cells in a common weight matrix, allowing consistent interpretation of factors across cells and yielding improved latent variable estimates. We compare FISHFactor to existing methods that rely on aggregating information through spatial binning and cannot combine information from multiple cells and show that our method leads to more accurate results on simulated data. We show that our method is scalable and can be readily applied to large datasets. Finally, we demonstrate on a real dataset that FISHFactor is able to identify major subcellular expression patterns and spatial gene clusters in a data-driven manner. The model implementation, data simulation and experiment scripts are available under https://www.github.com/bioFAM/FISHFactor.

Fast and Accurate Non-Negative Latent Factor Analysis of High-Dimensional and Sparse Matrices in Recommender Systems

A fast non-negative latent factor (FNLF) model for a high-dimensional and sparse (HiDS) matrix adopts a Single Latent Factor-dependent, Non-negative, Multiplicative and Momentum-incorporated Update (SLF-NM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> U) algorithm, which enables its fast convergence. It is crucial to achieve a rigorously theoretical proof regarding its fast convergence, which has not been provided in prior research. Aiming at addressing this critical issue, this work theoretically proves that with an appropriately chosen momentum coefficient, SLF-NM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> U enables the fast convergence of an FNLF model in both continuous and discrete time cases. Empirical analysis of HiDS matrices generated by representative industrial applications provides empirical evidences for the theoretical proof. Hence, this study represents an important milestone in the field of HiDS matrix analysis.

Obtaining clean and informative mass spectra from complex chromatographic and high-resolution all-ions-fragmentation data by nonnegative parallel factor analysis 2

A major challenge in processing of complex data obtained from chromatography hyphenated to mass spectrometry is to resolve chromatographically co-eluting compounds. In this study, we present a workflow for the resolution of ultra-high pressure liquid chromatography high-resolution mass spectrometry data obtained by the broadband data-independent acquisition MSE operation (UHPLCHRMSE). The workflow is based on a recently introduced algorithm for Parallel Factor Analysis 2 (PARAFAC2) that allows to enforce non-negativity on all the model coefficients. The workflow was tested on three sets of UHPLC-HRMSE measurements from a Lupinus angustifolius L. crop field study, which included plant tissue samples, soil samples and samples from drainage water as well as stream water close to the field. The three datasets included 93, 59, and 75 chromatographic runs in total for the plant, soil and water batches, respectively.Nonnegative-PARAFAC2 models were fitted on the summed high and low energy (HE and LE) traces on chromatographic intervals corresponding to spiked standard for the three sample sets independently. In soil and plant samples, 13 out of 14 spiked standards were resolved by NN-PARAFAC2 even in presence of chromatographic co-elution, and their mass spectral loadings could be matched to a reference spectrum. In contrast, only seven spiked standards were correctly resolved and matched for the water samples because a higher chromatographic baseline rendered the data noisier.The results show that the workflow we present can provide improved mass spectral selectivity for data-independent acquisition compared to using the raw mass spectra and can be used to match fragment ions from the HE trace, and precursor and adduct ions from the LE trace even in presence of co-eluting compounds.

Revealing Physiochemical Factors and Zooplankton Influencing Microcystis Bloom Toxicity in a Large-Shallow Lake Using Bayesian Machine Learning.

Toxic cyanobacterial blooms have become a severe global hazard to human and environmental health. Most studies have focused on the relationships between cyanobacterial composition and cyanotoxins production. Yet, little is known about the environmental conditions influencing the hazard of cyanotoxins. Here, we analysed a unique 22 sites dataset comprising monthly observations of water quality, cyanobacterial genera, zooplankton assemblages, and microcystins (MCs) quota and concentrations in a large-shallow lake. Missing values of MCs were imputed using a non-negative latent factor (NLF) analysis, and the results achieved a promising accuracy. Furthermore, we used the Bayesian additive regression tree (BART) to quantify how Microcystis bloom toxicity responds to relevant physicochemical characteristics and zooplankton assemblages. As expected, the BART model achieved better performance in Microcystis biomass and MCs concentration predictions than some comparative models, including random forest and multiple linear regression. The importance analysis via BART illustrated that the shade index was overall the best predictor of MCs concentrations, implying the predominant effects of light limitations on the MCs content of Microcystis. Variables of greatest significance to the toxicity of Microcystis also included pH and dissolved inorganic nitrogen. However, total phosphorus was found to be a strong predictor of the biomass of total Microcystis and toxic M. aeruginosa. Together with the partial dependence plot, results revealed the positive correlations between protozoa and Microcystis biomass. In contrast, copepods biomass may regulate the MC quota and concentrations. Overall, our observations arouse universal demands for machine-learning strategies to represent nonlinear relationships between harmful algal blooms and environmental covariates.

An α-β-Divergence-Generalized Recommender for Highly Accurate Predictions of Missing User Preferences.

To quantify user-item preferences, a recommender system (RS) commonly adopts a high-dimensional and sparse (HiDS) matrix. Such a matrix can be represented by a non-negative latent factor analysis model relying on a single latent factor (LF)-dependent, non-negative, and multiplicative update algorithm. However, existing models' representative abilities are limited due to their specialized learning objective. To address this issue, this study proposes an α- β -divergence-generalized model that enjoys fast convergence. Its ideas are three-fold: 1) generalizing its learning objective with α- β -divergence to achieve highly accurate representation of HiDS data; 2) incorporating a generalized momentum method into parameter learning for fast convergence; and 3) implementing self-adaptation of controllable hyperparameters for excellent practicability. Empirical studies on six HiDS matrices from real RSs demonstrate that compared with state-of-the-art LF models, the proposed one achieves significant accuracy and efficiency gain to estimate huge missing data in an HiDS matrix.

Non-negative Independent Factor Analysis disentangles discrete and continuous sources of variation in scRNA-seq data.

Single-cell RNA-seq analysis has emerged as a powerful tool for understanding inter-cellular heterogeneity. Due to the inherent noise of the data, computational techniques often rely on dimensionality reduction (DR) as both a pre-processing step and an analysis tool. Ideally, DR should preserve the biological information while discarding the noise. However, if the DR is to be used directly to gain biological insight it must also be interpretable-that is the individual dimensions of the reduction should correspond to specific biological variables such as cell-type identity or pathway activity. Maximizing biological interpretability necessitates making assumption about the data structures and the choice of the model is critical. We present a new probabilistic single-cell factor analysis model, Non-negative Independent Factor Analysis (NIFA), that incorporates different interpretability inducing assumptions into a single modeling framework. The key advantage of our NIFA model is that it simultaneously models uni- and multi-modal latent factors, and thus isolates discrete cell-type identity and continuous pathway activity into separate components. We apply our approach to a range of datasets where cell-type identity is known, and we show that NIFA-derived factors outperform results from ICA, PCA, NMF and scCoGAPS (an NMF method designed for single-cell data) in terms of disentangling biological sources of variation. Studying an immunotherapy dataset in detail, we show that NIFA is able to reproduce and refine previous findings in a single analysis framework and enables the discovery of new clinically relevant cell states. NFIA is a R package which is freely available at GitHub (https://github.com/wgmao/NIFA). The test dataset is archived at https://zenodo.org/record/6286646. Supplementary data are available at Bioinformatics online.

Factorization Problems on Rational Loop Groups, and the Poisson Geometry of Yang-Baxter Maps

Precise representation of large-scale undirected network is the basis for understanding relations within a massive entity set. The undirected network representation task can be efficiently addressed by a symmetry non-negative latent factor (SNLF) model, whose objective is clearly non-convex. However, existing SNLF models commonly adopt a first-order optimizer that cannot well handle the non-convex objective, thereby resulting in inaccurate representation results. On the other hand, higher-order learning algorithms are expected to make a breakthrough, but their computation efficiency are greatly limited due to the direct manipulation of the Hessian matrix, which can be huge in undirected network representation tasks. Aiming at addressing this issue, this study proposes to incorporate an efficient second-order method into SNLF, thereby establishing a second-order symmetric non-negative latent factor analysis model for undirected network with two-fold ideas: a) incorporating a mapping strategy into SNLF model to form an unconstrained model, and b) training the unconstrained model with a specially designed second order method to acquire a proper second-order step efficiently. Empirical studies indicate that proposed model outperforms state-of-the-art models in representation accuracy with affordable computational burden.

An Instance-Frequency-Weighted Regularization Scheme for Non-Negative Latent Factor Analysis on High-Dimensional and Sparse Data

High-dimensional and sparse (HiDS) data with non-negativity constraints are commonly seen in industrial applications, such as recommender systems. They can be modeled into an HiDS matrix, from which non-negative latent factor analysis (NLFA) is highly effective in extracting useful features. Preforming NLFA on an HiDS matrix is ill-posed, desiring an effective regularization scheme for avoiding overfitting. Current models mostly adopt a standard L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> scheme, which does not consider the imbalanced distribution of known data in an HiDS matrix. From this point of view, this paper proposes an instance-frequency-weighted regularization (IR) scheme for NLFA on HiDS data. It specifies the regularization effects on each latent factors with its relevant instance count, i.e., instance-frequency, which clearly describes the known data distribution of an HiDS matrix. By doing so, it achieves finely grained modeling of regularization effects. The experimental results on HiDS matrices from industrial applications demonstrate that compared with an L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> scheme, an IR scheme enables a resultant model to achieve higher accuracy in missing data estimation of an HiDS matrix.

Generalized Nesterov's Acceleration-Incorporated, Non-Negative and Adaptive Latent Factor Analysis

A non-negative latent factor (NLF) model with a single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) algorithm is frequently adopted to extract useful knowledge from non-negative data represented by high-dimensional and sparse (HiDS) matrices arising from various service-oriented applications. However, its convergence rate is slow. To address this issue, this study proposes a <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</u> eneralized Nesterov's acceleration-incorporated, <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</u> on-negative and <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u> daptive <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</u> atent <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</u> actor (GNALF) model. It results from a) incorporating a generalized Nesterov's accelerated gradient (NAG) method into an SLF-NMU algorithm, thereby achieving an <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</u> AG-incorporated and <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e</u> lement-oriented <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</u> on-negative (NEN) algorithm to perform efficient parameter update; and b) making its regularization and acceleration parameters self-adaptive via incorporating the principle of a particle swarm optimization algorithm into the training process, thereby implementing a highly adaptive and practical model. Empirical studies on six large sparse matrices from different recommendation service applications show that a GNALF model achieves very high convergence rate without the need of hyper-parameter tuning, making its computational efficiency significantly higher than state-of-the-art models. Meanwhile, such efficiency gain does not result in accuracy loss, since its prediction accuracy is comparable with its peers. Hence, it can better serve practical service applications with real-time demands.

Algorithms of Unconstrained Non-Negative Latent Factor Analysis for Recommender Systems

Non-negativity is vital for a latent factor (LF)-based model to preserve the important feature of a high-dimensional and sparse (HiDS) matrix in recommender systems, i.e., none of its entries is negative. Current non-negative models rely on constraints-combined training schemes. However, they lack flexibility, scalability, or compatibility with general training schemes. This work aims to perform unconstrained non-negative latent factor analysis (UNLFA) on HiDS matrices. To do so, we innovatively transfer the non-negativity constraints from the decision parameters to the output LFs, and connect them through a single-element-dependent mapping function. Then we theoretically prove that by making a mapping function fulfill specific conditions, the resultant model is able to represent the original one precisely. We subsequently design highly efficient UNLFA algorithms for recommender systems. Experimental results on four industrial-size HiDS matrices demonstrate that compared with four state-of-the-art non-negative models, a UNLFA-based model obtains advantage in prediction accuracy for missing data and computational efficiency. Moreover, such high performance is achieved through its unconstrained training process which is compatible with various general training schemes, on the premise of fulfilling non-negativity constraints. Hence, UNLFA algorithms are highly valuable for industrial applications with the need of performing non-negative latent factor analysis on HiDS matrices.

Bayesian multivariate receptor modeling software: BNFA and bayesMRM

We present user-friendly software tools to implement Bayesian multivariate receptor modeling in the form of a MATLAB function (BNFA) and an R package (bayesMRM). A basic model and a Markov chain Monte Carlo algorithm underlying BNFA and bayesMRM are given. An example of implementation based on real air pollution data is also provided. Users can freely choose between BNFA and bayesMRM depending on their computing platform. These tools are expected to facilitate implementation of Bayesian multivariate receptor models and/or Bayesian nonnegative factor analysis models and promote their use in chemometrics.

Assimilating Second-Order Information for Building Non-Negative Latent Factor Analysis-Based Recommenders

A non-negative latent factor analysis (NLFA)-based recommender can make precise recommendations by correctly representing the non-negative characteristic of industrial data. It commonly relies on a nonconvex and bilinear optimization process, where the effects of first-order solvers maybe significantly reduced. Higher order solvers like a Newton-type method are expected to make a breakthrough; however, its computation efficiency and scalability are greatly limited due to the numerous parameters involved in a Hessian matrix. To address this issue, this article proposes an approach for assimilating second-order information for building NLFA-based recommenders. The key idea is an inner second-order solver that employs a Hessian-free method for avoiding the highly expensive manipulations of a Hessian matrix. Empirical studies on eight data cases emerging from real industrial applications indicate that the proposed approach outperforms state-of-the-art models in prediction accuracy with affordable computational burden.

A composite indicator for the waste management in the EU via Hierarchical Disjoint Non-Negative Factor Analysis

In the last years, the quantity of information and statistics about waste management are more and more consistent but so far, few studies are available in this field. The goal of this paper is of producing a model-based Composite Indicator of “good” Waste Management, in order to provide a useful tool of support for EU countries’ policy-makers and institutions.Composite Indicators (CIs), usually, are multidimensional concepts with a hierarchical structure characterized by the presence of a set of specific dimensions, each one corresponding to a subsets of manifest variables. Thus, we propose a CI for Waste Management in Europe by using a hierarchical model-based approach with positive loadings. This approach guarantees to comply with all the good properties on which a composite indicator should be based and to detect the main dimensions (i.e., aspects) of the Waste Management phenomenon.In other terms, this paper provides a hierarchically aggregated index that best describes the Waste Management in EU with its main features by identifying the most important high order (i.e., hierarchical) relationships among subsets of manifest variables. All the parameters are estimated according to the maximum likelihood estimation method (MLE) in order to make inference on the parameters and on the validity of the model.

Statistical Model-Based Composite Indicators for Tracking Coherent Policy Conclusions

In the last years, with the data revolution and the use of new technologies, phenomena are frequently described by a huge quantity of information useful for making strategic decisions. A priority for policymakers is having simple statistical tools useful to synthesize data. Such tools are represented by composite indicators (CIs). According to the glossary of statistical terms of OECD (The OECD-JRC handbook on practices for developing composite indicators. Paper presented at the OECD Committee on Statistics, 2004), OECD-JRC (Handbook on constructing composite indicators. Methodology and user guide, OECD, Paris, 2008), a CI is formed when manifest (observed) indicators are compiled into a single index, on the basis of an underlying model of the multi-dimensional concept that is being measured, and weights commonly represent the relative importance of each indicator. CIs are increasingly used for bench-marking countries’ performances and the methodological challenges raise a series of technical issues that, if not adequately addressed, can lead to CIs being misinterpreted or manipulated. Yet doubts are often raised about the robustness of the resulting countries’ rankings and about the significance of the associated policy message. In this paper, we propose a model-based approach for the construction of CIs with a hierarchical structure where the CIs (first and second order) are estimated using the Hierarchical Disjoint Non-Negative Factor Analysis (Cavicchia and Vichi in Hierarchical disjoint non-negative factor analysis. Manuscript submitted for publication, 2020) in a LS framework. In order to assess the methodology of construction of a CI, a set of properties is proposed and applied. Some well-known CIs, such as the Human Development Index and the Multidimensional Poverty index, are taken into consideration to show the importance of those properties. Therefore, we include into our proposal the most frequently used approaches in the literature of CIs, and we evaluate the model to assess their performances.

Genomic data integration by WON-PARAFAC identifies interpretable factors for predicting drug-sensitivity in vivo

Integrative analyses that summarize and link molecular data to treatment sensitivity are crucial to capture the biological complexity which is essential to further precision medicine. We introduce Weighted Orthogonal Nonnegative parallel factor analysis (WON-PARAFAC), a data integration method that identifies sparse and interpretable factors. WON-PARAFAC summarizes the GDSC1000 cell line compendium in 130 factors. We interpret the factors based on their association with recurrent molecular alterations, pathway enrichment, cancer type, and drug-response. Crucially, the cell line derived factors capture the majority of the relevant biological variation in Patient-Derived Xenograft (PDX) models, strongly suggesting our factors capture invariant and generalizable aspects of cancer biology. Furthermore, drug response in cell lines is better and more consistently translated to PDXs using factor-based predictors as compared to raw feature-based predictors. WON-PARAFAC efficiently summarizes and integrates multiway high-dimensional genomic data and enhances translatability of drug response prediction from cell lines to patient-derived xenografts.

Non-negative Factor (NNF) Assisted Partial Least Square (PLS) Analysis of Excitation-Emission Matrix Fluorescence Spectroscopic Data Sets: Automating the Identification and Quantification of Multifluorophoric Mixtures.

Excitation-emission matrix fluorescence spectroscopy is simple and sensitive techniques that generate the composite fluorescence fingerprints. EEMF can be used for the identification and quantification of the fluorophores without involving any pre-separation step provided a suitable data analysis approach is applied. In the present work, non-negative factor (NNF) assisted partial least square (PLS) analysis is used for the analysis of EEMF data sets acquired for the dilute aqueous mixtures of fluorophores. The proposed approach allows automatic selection of the optimum number of factors for NNF analysis by incorporating the Akaike information criterion. The proposed approach also incorporates the spectral correlation analysis for the automatic identification of the NNF retrieved EEMF spectral profiles. The NNF retrieved contribution values along with their real concentration values are subjected to PLS analysis to develop a calibration model. The proposed approach was successfully tested using EEMF data acquired for the dilute aqueous mixtures of Catechol, Hydroquinone, Indole, Tryptophan and Tyrosine. The results were evaluated using the various statistical parameters and each of them found to well within the expected limits. In summary, NNF assisted PLS analysis of EEMF technique allows automatized analysis of the multifluorophoric mixtures with minimum user inputs.

Chemometric assisted Fourier Transform Infrared (FTIR) Spectroscopic analysis of fruit wine samples: Optimizing the initialization and convergence criteria in the non-negative factor analysis algorithm for developing a robust classification model.

The present work proposes certain optimization in the non-negative factor analysis (NNFA) algorithm to ensure an efficient analysis of the Fourier transformation infrared (FTIR) spectral data sets of the fruit wine samples. The first optimization deals with initialization of the variables in a controlled fashion that would ensure a reasonably good quality initial estimate to implement NNFA algorithm. It prevents NNFA algorithm from itinerating with random numbers that essentially have no chemical relevance. The second implemented optimization involves eliminating the alternate least square of convergence and allowing the algorithm to iterate until the iteration limit is reached. This criterion avoids the algorithm to have premature convergence and ensures that model provide the solutions which corresponds to the global minima. The application of NNFA with suggested optimizations are found to capture the subtle differences in the spectral profiles and classify the fruit wine samples that are essentially complex mixtures of several chemicals in unknown proportions. The proposed approach is also found to perform better than principal component analysis on practical grounds. In summary, the current work provides a simple, sensitive and cost-effective approach using optimized NNFA and FTIR spectroscopy for classifying the fruit wine samples.