Non-negative Factor Research Articles

Alternating nonnegative least squares-incorporated regularized symmetric latent factor analysis for undirected weighted networks

An Undirected Weighted Network (UWN) can be precisely quantified as an adjacency matrix whose inherent characteristics are fully considered in a Symmetric Nonnegative Latent Factor (SNLF) model for its good representation accuracy. However, an SNLF model uses a sole latent factor matrix to precisely describe the topological characteristic of a UWN, i.e., symmetry, thereby impairing its representation learning ability. Aiming at addressing this issue, this paper proposes an Alternating nonnegative least squares-incorporated Regularized Symmetric Latent factor analysis (ARSL) model. First of all, equation constraints composed of multiple matrices are built in its learning objective for well describing the symmetry of a UWN. Note that it adopts an L2-norm-based regularization scheme to relax such constraints for making such a symmetry-aware learning objective solvable. Then, it designs an alternating nonnegative least squares-incorporated algorithm for optimizing its parameters efficiently. Empirical studies on four UWNs demonstrate that an ARSL model outperforms the state-of-the-art models in terms of representation accuracy, as well as achieves promising computational efficiency.

Electricity Retail Plan Recommendation Method Based on Multigranular Hesitant Fuzzy Sets and an Improved Non-Negative Latent Factor Model

Electricity Retail Plan Recommendation Method Based on Multigranular Hesitant Fuzzy Sets and an Improved Non-Negative Latent Factor Model

Switching Triple-Weight-SMOTE in Empirical Feature Space for Imbalanced and Incomplete Data

Most existing techniques to handle imbalanced data might be invalid in the presence of data missing since they are based on the assumption that the data is complete. To shorten such a gap, a novel synthetic minority oversampling technique (SMOTE), i.e., Non-negative latent factor analysis-incorporated and Switching triple-weight-SMOTE (NSS), is proposed. The main idea of NSS is 4-fold: 1) a Lagrange non-negative matrix factorization (LNMF) method is put forward to impute the missing values with a guaranteed non-negativity according to the original distribution owing to the consideration of the global feature information; 2) by mapping the complete data after imputation into an empirical feature space (EFS), a more separable dataset is achieved, which rigidly maintains the same geometrical structure as the original data while efficiently reducing the redundant features to enhance model generalization and operational efficiency; 3) after fulfilling the fuzzy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$c$</tex-math> </inline-formula> -means (FCM) clustering, the inter-cluster distance, the capacity of each minority cluster and its sparsity are comprehensively taken into account to develop a triple-weight assignment strategy, which contributes to allocate the number of synthetic samples to each cluster appropriately; 4) a switching oversampling strategy is provided in response to the clusters with different distributions (i.e., either Gaussian or uniform distribution). Moreover, a posterior is used to check the correctness of the synthetic samples. Finally, experiments on a real dataset and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$12$</tex-math> </inline-formula> public datasets show that the proposed NSS outperforms other <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$11$</tex-math> </inline-formula> state-of-art methods. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Data classification is an important computer task, which has been successfully used in many domains, including but not limited to, the medical domain, finance domain, and manufacturing domain. However, there are two big challenges when a classifier is to handle real-world data in the presence of imbalanced data and missing values. More specifically, the model performance is very likely to be degraded due to the missing information and the inclination of classifiers to the majority class. To surmount this problem, a natural idea is to use the LNMF model to obtain the desired recovery. Then, for the complete data after imputation, the empirical-feature-space-based switching triple-weight-SMOTE is applied to synthesize safe and correct data (i.e., lie solidly on the region of minority class) to achieve the balance. Such a working principle generates a novel NSS strategy. The proposed NSS strategy has the following obvious merits: 1) the imputation guarantees the similarity to the original dataset; and 2) new synthetic data are safely generated by taking adequate consideration of the information and distribution of datasets. Thus, the proposed NSS can greatly help improve the classification accuracy of real-world datasets.

Adaptive Divergence-Based Non-Negative Latent Factor Analysis of High-Dimensional and Incomplete Matrices From Industrial Applications

Adaptive Divergence-Based Non-Negative Latent Factor Analysis of High-Dimensional and Incomplete Matrices From Industrial Applications

An Adaptive Divergence-Based Non-Negative Latent Factor Model

A High-dimensional and incomplete (HDI) matrix is regularly adopted to portray the inherent non-negativity of interactions among numerous nodes, which is involved in countless industrial applications driven by big data. An inherently non-negative latent factor (LF) model can take out the intrinsical features from such data conveniently and effectually due to its unimpeded training process. However, it constructs the learning objective relying on a standard Euclidean distance, thereby seriously restricting its representative ability to HDI data generated by different domains. To address this issue, this work proposes an adaptive divergence-based non-negative LF (ADNLF) model following: 1) constructing a generalized objective function based on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <inline-formula> <tex-math notation="LaTeX">$\alpha$</tex-math> </inline-formula>-<inline-formula> <tex-math notation="LaTeX">$\beta$</tex-math> </inline-formula> </i> -divergence to inflate its ability to represent various HDI data; 2) connecting the optimization variables with output LFs by a smooth and single LF-dependent bridging function to satisfy the non-negativity constraints constantly; and 3) facilitating adaptive divergence in the learning objective through particle swarm optimization for high scalability. Empirical studies on eight HDI matrices validate that an ADNLF model evidently outstrips state-of-the-art models in terms of estimation accuracy as well as computational efficiency for missing data of an HDI dataset.

An Improved Non-Negative Latent Factor Model for Missing Data Estimation via Extragradient-Based Alternating Direction Method.

In this article, an improved double factorization-based symmetric and non-negative latent factor (Im-DF-SNLF) model is proposed to make the estimation for missing data in symmetric, high-dimensional, and sparse (SHiDS) matrices. The main idea of the Im-DF-SNLF model is fourfold: 1) considering the data variety in the practical engineering, non-negative latent factors (NLFs) in different cases are considered to better reflect the latent relationships between entries; 2) the l₂-norm regularization and the Lagrangian multiplier technique are simultaneously adopted to handle the overfitting and satisfy the non-negative constraint for latent factors (LFs); 3) the extragradient-based alternating direction (EGAD) method is utilized to accelerate the model training and rigidly guarantee the non-negativity of LFS; and 4) a rigorous proof is provided to show that, under the given assumption that the objective function is smooth and has a Lipschitz continuous gradient, the designed algorithm can find an ε-optimal solution within O(1/ε), and the upper bound of the learning rate is given by 1/2. Finally, experimental results on public datasets are given to demonstrate the effectiveness of our proposed Im-DF-SNLF model with EGAD.

Non-target ROIMCR LC–MS analysis of the disruptive effects of TBT over time on the lipidomics of Daphnia magna

IntroductionThis study has investigated the temporal disruptive effects of tributyltin (TBT) on lipid homeostasis in Daphnia magna. To achieve this, the study used Liquid Chromatography–Mass Spectrometry (LC–MS) analysis to analyze biological samples of Daphnia magna treated with TBT over time. The resulting data sets were multivariate and three-way, and were modeled using bilinear and trilinear non-negative factor decomposition chemometric methods. These methods allowed for the identification of specific patterns in the data and provided insight into the effects of TBT on lipid homeostasis in Daphnia magna.ObjectivesInvestigation of how are the changes in the lipid concentrations of Daphnia magna pools when they were exposed with TBT and over time using non-targeted LC–MS and advanced chemometric analysis.MethodsThe simultaneous analysis of LC–MS data sets of Daphnia magna samples under different experimental conditions (TBT dose and time) were analyzed using the ROIMCR method, which allows the resolution of the elution and mass spectra profiles of a large number of endogenous lipids. Changes obtained in the peak areas of the elution profiles of these lipids caused by the dose of TBT treatment and the time after its exposure are analyzed by principal component analysis, multivariate curve resolution-alternative least square, two-way ANOVA and ANOVA-simultaneous component analysis.Results87 lipids were identified. Some of these lipids are proposed as Daphnia magna lipidomic biomarkers of the effects produced by the two considered factors (time and dose) and by their interaction. A reproducible multiplicative effect between these two factors is confirmed and the optimal approach to model this dataset resulted to be the application of the trilinear factor decomposition model.ConclusionThe proposed non-targeted LC–MS lipidomics approach resulted to be a powerful tool to investigate the effects of the two factors on the Daphnia magna lipidome using chemometric methods based on bilinear and trilinear factor decomposition models, according to the type of interaction between the design factors.

An Alternating-Direction-Method of Multipliers-Incorporated Approach to Symmetric Non-Negative Latent Factor Analysis.

Large-scale undirected weighted networks are frequently encountered in big-data-related applications concerning interactions among a large unique set of entities. Such a network can be described by a Symmetric, High-Dimensional, and Incomplete (SHDI) matrix whose symmetry and incompleteness should be addressed with care. However, existing models fail in either correctly representing its symmetry or efficiently handling its incomplete data. For addressing this critical issue, this study proposes an Alternating-Direction-Method of Multipliers (ADMM)-based Symmetric Non-negative Latent Factor Analysis (ASNL) model. It adopts fourfold ideas: 1) implementing the data density-oriented modeling for efficiently representing an SHDI matrix's incomplete and imbalanced data; 2) separating the non-negative constraints from the decision parameters to avoid truncations during the training process; 3) incorporating the ADMM principle into its learning scheme for fast model convergence; and 4) parallelizing the training process with load balance considerations for high efficiency. Empirical studies on four SHDI matrices demonstrate that ASNL significantly outperforms several state-of-the-art models in both prediction accuracy for missing data of an SHDI and computational efficiency. It is a promising model for handling large-scale undirected networks raised in real applications.

Accurate regularized Tucker decomposition for image restoration

We propose a new accurate regularized Tucker decomposition (ARTD) method for image restoration (IR), which considers global low-rankness and local similarity of intrinsic image characteristics. Specifically, global low-rankness is represented by a sparse Tucker core tensor, whereas the local similarity is captured using nonnegative factor matrices and manifold regularization terms. Sparse nonnegative Tucker decomposition (SNTD) and graph nonnegative Tucker decomposition (GNTD) can be considered a special case of ARTD. We propose and implement an effective Alternating Proximal Gradient (APG) based algorithm to solve the ARTD model and deduce the closed-form updating rules. Notably, ARTD does not need to tune the Tucker rank and provides an initialization strategy and a first-order feedback control rule to accelerate its convergence. Experiments on real IR problems show that our method outperforms some existing state-of-the-art methods.

Proximal Alternating-Direction-Method-of-Multipliers-Incorporated Nonnegative Latent Factor Analysis

High-dimensional and incomplete (HDI) data subject to the nonnegativity constraints are commonly encountered in a big data-related application concerning the interactions among numerous nodes. A nonnegative latent factor analysis (NLFA) model can perform representation learning to HDI data efficiently. However, existing NLFA models suffer from either slow convergence rate or representation accuracy loss. To address this issue, this paper proposes a proximal alternating-direction-method-of-multipliers-based nonnegative latent factor analysis (PAN) model with two-fold ideas: 1) adopting the principle of alternating-direction-method-of-multipliers to implement an efficient learning scheme for fast convergence and high computational efficiency; and 2) incorporating the proximal regularization into the learning scheme to suppress the optimization fluctuation for high representation learning accuracy to HDI data. Theoretical studies verify that PAN converges to a Karush-Kuhn-Tucker (KKT) stationary point of its nonnegativity-constrained learning objective with its learning scheme. Experimental results on eight HDI matrices from real applications demonstrate that the proposed PAN model outperforms several state-of-the-art models in both estimation accuracy for missing data of an HDI matrix and computational efficiency.

FISHFactor: a probabilistic factor model for spatial transcriptomics data with subcellular resolution.

Factor analysis is a widely used tool for unsupervised dimensionality reduction of high-throughput datasets in molecular biology, with recently proposed extensions designed specifically for spatial transcriptomics data. However, these methods expect (count) matrices as data input and are therefore not directly applicable to single molecule resolution data, which are in the form of coordinate lists annotated with genes and provide insight into subcellular spatial expression patterns. To address this, we here propose FISHFactor, a probabilistic factor model that combines the benefits of spatial, non-negative factor analysis with a Poisson point process likelihood to explicitly model and account for the nature of single molecule resolution data. In addition, FISHFactor shares information across a potentially large number of cells in a common weight matrix, allowing consistent interpretation of factors across cells and yielding improved latent variable estimates. We compare FISHFactor to existing methods that rely on aggregating information through spatial binning and cannot combine information from multiple cells and show that our method leads to more accurate results on simulated data. We show that our method is scalable and can be readily applied to large datasets. Finally, we demonstrate on a real dataset that FISHFactor is able to identify major subcellular expression patterns and spatial gene clusters in a data-driven manner. The model implementation, data simulation and experiment scripts are available under https://www.github.com/bioFAM/FISHFactor.

Fast and Accurate Non-Negative Latent Factor Analysis of High-Dimensional and Sparse Matrices in Recommender Systems

A fast non-negative latent factor (FNLF) model for a high-dimensional and sparse (HiDS) matrix adopts a Single Latent Factor-dependent, Non-negative, Multiplicative and Momentum-incorporated Update (SLF-NM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> U) algorithm, which enables its fast convergence. It is crucial to achieve a rigorously theoretical proof regarding its fast convergence, which has not been provided in prior research. Aiming at addressing this critical issue, this work theoretically proves that with an appropriately chosen momentum coefficient, SLF-NM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> U enables the fast convergence of an FNLF model in both continuous and discrete time cases. Empirical analysis of HiDS matrices generated by representative industrial applications provides empirical evidences for the theoretical proof. Hence, this study represents an important milestone in the field of HiDS matrix analysis.

A Second-Order Symmetric Non-Negative Latent Factor Model for Undirected Weighted Network Representation

Precise representation to undirected weighted network (UWN) is the foundation of understanding connection patterns inside a massive node set. It can be addressed via a Symmetric Non-negative Latent Factor (SNLF) model with a non-convex learning objective. However, existing SNLF models commonly adopt a first-order learning algorithm that cannot well handle such a non-convex objective, thereby leading to inaccurate UWN representation. Aiming at addressing this issue, this study incorporates an efficient second-order learning algorithm into an SNLF model, thereby establishing a Second-order Symmetric Non-negative Latent Factor (S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> NLF) model with two-fold ideas: a) applying the single latent factor-related mapping function to the non-negativity constrained optimization parameters to achieve an unconstrained learning objective, and b) optimizing this learning objective with its optimization parameters through an efficient second-order learning algorithm to achieve accurate representation to the target UWN with affordable computational burden. Empirical studies indicate that owing to its efficient incorporation of the second-order optimization technique, the proposed S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> NLF model outperforms state-of-the-art SNLF models when they are used to gain highly accurate representation to UWNs emerging from real applications.

Accelerating probabilistic tensor canonical polyadic decomposition with nonnegative factors: An inexact BCD Approach

Recently, Bayesian modeling and variational inference (VI) were leveraged to enable the nonnegative factor matrix learning with automatic rank determination in tensor canonical polyadic decomposition (CPD), which has found various applications in big data analytics. However, since VI inherently performs block coordinate descent (BCD) steps over the functional space, it generally does not allow integration with modern large-scale optimization methods, making the scalability a critical issue. In this paper, it is revealed that the expectations of the variables updated by the VI algorithm is equivalent to the block minimization steps of a deterministic optimization problem. This equivalence further enables the adoption of inexact BCD method for devising a fast nonnegative factor matrix learning algorithm with automatic tensor rank determination. Numerical results using synthetic data and real-world applications show that the performance of the proposed algorithm is comparable with that of the VI-based algorithm, but with computation times reduced significantly.

Layered solutions for a nonlocal Ginzburg-Landau model with periodic modulation

&lt;abstract&gt;&lt;p&gt;We study layered solutions in a one-dimensional version of the scalar Ginzburg-Landau equation that involves a mixture of a second spatial derivative and a fractional half-derivative, together with a periodically modulated nonlinearity. This equation appears as the Euler-Lagrange equation of a suitably renormalized fractional Ginzburg-Landau energy with a double-well potential that is multiplied by a 1-periodically varying nonnegative factor $ g(x) $ with $ \int_0^1 \frac{1}{g(x)} dx &amp;lt; \infty. $ A priori this energy is not bounded below due to the presence of a nonlocal term in the energy. Nevertheless, through a careful analysis of a minimizing sequence we prove existence of global energy minimizers that connect the two wells at infinity. These minimizers are shown to be the classical solutions of the associated nonlocal Ginzburg-Landau type equation.&lt;/p&gt;&lt;/abstract&gt;

Nonnegative Latent Factor Analysis-Incorporated and Feature-Weighted Fuzzy Double $c$-Means Clustering for Incomplete Data

Fuzzy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$c$</tex-math></inline-formula> -means (FCM) clustering is a promising method to handle uncertainties in data clustering. However, the traditional FCM and most of its variants cannot address incomplete inputs. To this aim, a novel fuzzy clustering framework is put forward to perform highly accurate clustering on incomplete data. It adopts twofold ideas: 1) Utilizing a nonnegative latent factor model to prefill the missing data in the inputs by rigidly extracting involved entities’ latent features, where the principle of a minibatch gradient descent algorithm is incorporated into a single latent factor-dependent, nonnegative and multiplicative update algorithm to accelerate the convergence rate; and 2) integrating the distribution of inputs and the weights of local features into the objective function through sparse self-representation and weighting allocation to focus on crucial features. In this way, a NLF analysis-incorporated and feature-weighted fuzzy double <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$c$</tex-math></inline-formula> -means clustering (NF <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^2$</tex-math></inline-formula> D) method is achieved, where the data distribution and instance correlation are simultaneously considered with care. Experiments on 12 real-world datasets including both data and images with different missing rates show that the proposed NF <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^2$</tex-math></inline-formula> D method has a significant superiority over state-of-the-art fuzzy clustering methods.

Triple Factorization-Like Symmetric NLF Models With Latent Item–Item Relationship

Undirected, high-dimension and sparse (SHiDS) networks which are often encountered in industrial applications usually contain numerous useful information; thus, it is practically important to develop effective collaborative filtering methods to rigidly extract the latent information. In order to improve the performance of nonnegative latent factor (NLF) models based on the double factorization while reducing the computation burden of NLF models in terms of the triple factorization (TF) technique, a novel symmetric NLF (SNLF) model, i.e., TF-like SNLF model, is proposed in this article. A latent factor (LF) matrix that reflects the relationship of items is introduced into the objective function so as to offer more freedom for solutions. Then, a fixed weighting matrix designed by sensitivity experiments is used to replace the aforementioned LF matrix dependent on the update, which greatly helps save the time cost without sacrificing any accuracy. Furthermore, the alteration direction (ADI) method is adopted to design the algorithm, which effectively helps reduce the occurrence of the local minimum while guaranteeing convergence. Finally, experimental results on four real industrial data sets demonstrate that the proposed TF-like SNLF model makes a good compromise between the model performance and the time efficiency.

Obtaining clean and informative mass spectra from complex chromatographic and high-resolution all-ions-fragmentation data by nonnegative parallel factor analysis 2

A major challenge in processing of complex data obtained from chromatography hyphenated to mass spectrometry is to resolve chromatographically co-eluting compounds. In this study, we present a workflow for the resolution of ultra-high pressure liquid chromatography high-resolution mass spectrometry data obtained by the broadband data-independent acquisition MSE operation (UHPLCHRMSE). The workflow is based on a recently introduced algorithm for Parallel Factor Analysis 2 (PARAFAC2) that allows to enforce non-negativity on all the model coefficients. The workflow was tested on three sets of UHPLC-HRMSE measurements from a Lupinus angustifolius L. crop field study, which included plant tissue samples, soil samples and samples from drainage water as well as stream water close to the field. The three datasets included 93, 59, and 75 chromatographic runs in total for the plant, soil and water batches, respectively.Nonnegative-PARAFAC2 models were fitted on the summed high and low energy (HE and LE) traces on chromatographic intervals corresponding to spiked standard for the three sample sets independently. In soil and plant samples, 13 out of 14 spiked standards were resolved by NN-PARAFAC2 even in presence of chromatographic co-elution, and their mass spectral loadings could be matched to a reference spectrum. In contrast, only seven spiked standards were correctly resolved and matched for the water samples because a higher chromatographic baseline rendered the data noisier.The results show that the workflow we present can provide improved mass spectral selectivity for data-independent acquisition compared to using the raw mass spectra and can be used to match fragment ions from the HE trace, and precursor and adduct ions from the LE trace even in presence of co-eluting compounds.

SAR Image Reconstruction of Vehicle Targets Based on Tensor Decomposition

Due to the imaging mechanism of Synthetic Aperture Radars (SARs), the target shape on an SAR image is sensitive to the radar incidence angle and target azimuth, but there is strong correlation and redundancy between adjacent azimuth images of SAR targets. This paper studies multi-angle SAR image reconstruction based on non-negative Tucker decomposition using adjacent azimuth images reconstructed to form a sparse tensor. Sparse tensors are used to perform non-negative Tucker decomposition, resulting in non-negative core tensors and factor matrices. The reconstruction tensor is obtained by calculating the n-mode product of the core tensor and the factor matrix, and then image reconstruction is realized. The similarity between the original image and the reconstructed image is calculated by using the structural similarity index and the cosine of the angle between the feature vectors. The reconstruction results of three target images of MSTAR show that the reconstructed image has a similarity higher than 95% with the original image in most cases, which can support target recognition under sparse observation to a certain extent.

An improved feature selection method for classification on incomplete data: Non-negative latent factor-incorporated duplicate MIC

Feature selection is an important data pre-processing procedure in the classification task, whose main purpose is to remove negligible/redundant features to reduce the computational cost, while improving the performance of the subsequent machine learning method. However, most feature selection methods can only work well with the complete data, but show poorly in the presence of the missing information, especially with a high missing rate, due probably to the severe irrelevance and redundancy of features. To address this issue, we proposed a novel feature selection method, namely non-negative latent factor incorporated duplicate maximal information coefficient (NLF-DMIC), which improves the effectiveness of feature selection for the classification of incomplete data. The NLF-DMIC method is fulfilled by the following three steps: (1) select category-friendly features by MIC based on “partial sample strategy” roughly; (2) use the NLF model to make an imputation for the missing data in terms of the filtered features; and (3) select features again by an improved maximal information coefficient (i.e., low-redundancy MIC, LMIC) method on the complete dataset. Finally, experiments on an artificially synthetic dataset and 8 real datasets show that the proposed NLF-DMIC method outperforms some state-of-the-art feature selection methods.