Non-negative Latent Factor Model Research Articles

Electricity Retail Plan Recommendation Method Based on Multigranular Hesitant Fuzzy Sets and an Improved Non-Negative Latent Factor Model

Electricity Retail Plan Recommendation Method Based on Multigranular Hesitant Fuzzy Sets and an Improved Non-Negative Latent Factor Model

An Adaptive Divergence-Based Non-Negative Latent Factor Model

A High-dimensional and incomplete (HDI) matrix is regularly adopted to portray the inherent non-negativity of interactions among numerous nodes, which is involved in countless industrial applications driven by big data. An inherently non-negative latent factor (LF) model can take out the intrinsical features from such data conveniently and effectually due to its unimpeded training process. However, it constructs the learning objective relying on a standard Euclidean distance, thereby seriously restricting its representative ability to HDI data generated by different domains. To address this issue, this work proposes an adaptive divergence-based non-negative LF (ADNLF) model following: 1) constructing a generalized objective function based on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <inline-formula> <tex-math notation="LaTeX">$\alpha$</tex-math> </inline-formula>-<inline-formula> <tex-math notation="LaTeX">$\beta$</tex-math> </inline-formula> </i> -divergence to inflate its ability to represent various HDI data; 2) connecting the optimization variables with output LFs by a smooth and single LF-dependent bridging function to satisfy the non-negativity constraints constantly; and 3) facilitating adaptive divergence in the learning objective through particle swarm optimization for high scalability. Empirical studies on eight HDI matrices validate that an ADNLF model evidently outstrips state-of-the-art models in terms of estimation accuracy as well as computational efficiency for missing data of an HDI dataset.

A Second-Order Symmetric Non-Negative Latent Factor Model for Undirected Weighted Network Representation

Precise representation to undirected weighted network (UWN) is the foundation of understanding connection patterns inside a massive node set. It can be addressed via a Symmetric Non-negative Latent Factor (SNLF) model with a non-convex learning objective. However, existing SNLF models commonly adopt a first-order learning algorithm that cannot well handle such a non-convex objective, thereby leading to inaccurate UWN representation. Aiming at addressing this issue, this study incorporates an efficient second-order learning algorithm into an SNLF model, thereby establishing a Second-order Symmetric Non-negative Latent Factor (S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> NLF) model with two-fold ideas: a) applying the single latent factor-related mapping function to the non-negativity constrained optimization parameters to achieve an unconstrained learning objective, and b) optimizing this learning objective with its optimization parameters through an efficient second-order learning algorithm to achieve accurate representation to the target UWN with affordable computational burden. Empirical studies indicate that owing to its efficient incorporation of the second-order optimization technique, the proposed S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> NLF model outperforms state-of-the-art SNLF models when they are used to gain highly accurate representation to UWNs emerging from real applications.

Triple Factorization-Like Symmetric NLF Models With Latent Item–Item Relationship

Undirected, high-dimension and sparse (SHiDS) networks which are often encountered in industrial applications usually contain numerous useful information; thus, it is practically important to develop effective collaborative filtering methods to rigidly extract the latent information. In order to improve the performance of nonnegative latent factor (NLF) models based on the double factorization while reducing the computation burden of NLF models in terms of the triple factorization (TF) technique, a novel symmetric NLF (SNLF) model, i.e., TF-like SNLF model, is proposed in this article. A latent factor (LF) matrix that reflects the relationship of items is introduced into the objective function so as to offer more freedom for solutions. Then, a fixed weighting matrix designed by sensitivity experiments is used to replace the aforementioned LF matrix dependent on the update, which greatly helps save the time cost without sacrificing any accuracy. Furthermore, the alteration direction (ADI) method is adopted to design the algorithm, which effectively helps reduce the occurrence of the local minimum while guaranteeing convergence. Finally, experimental results on four real industrial data sets demonstrate that the proposed TF-like SNLF model makes a good compromise between the model performance and the time efficiency.

Nonnegative Latent Factor Analysis-Incorporated and Feature-Weighted Fuzzy Double $c$-Means Clustering for Incomplete Data

Fuzzy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$c$</tex-math></inline-formula> -means (FCM) clustering is a promising method to handle uncertainties in data clustering. However, the traditional FCM and most of its variants cannot address incomplete inputs. To this aim, a novel fuzzy clustering framework is put forward to perform highly accurate clustering on incomplete data. It adopts twofold ideas: 1) Utilizing a nonnegative latent factor model to prefill the missing data in the inputs by rigidly extracting involved entities’ latent features, where the principle of a minibatch gradient descent algorithm is incorporated into a single latent factor-dependent, nonnegative and multiplicative update algorithm to accelerate the convergence rate; and 2) integrating the distribution of inputs and the weights of local features into the objective function through sparse self-representation and weighting allocation to focus on crucial features. In this way, a NLF analysis-incorporated and feature-weighted fuzzy double <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$c$</tex-math></inline-formula> -means clustering (NF <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^2$</tex-math></inline-formula> D) method is achieved, where the data distribution and instance correlation are simultaneously considered with care. Experiments on 12 real-world datasets including both data and images with different missing rates show that the proposed NF <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^2$</tex-math></inline-formula> D method has a significant superiority over state-of-the-art fuzzy clustering methods.

An enhanced matrix completion method based on non-negative latent factors for recommendation system

Among the model-based collaborative filtering (CF) recommendation algorithms, matrix factorization (MF) technology is quite efficient. An ever-increasing focus has witnessed that the non-negative latent factor (NLF)-based MF model is superior to other state-of-art models, due to its great ability to guarantee a desirable prediction accuracy while grasping the non-negativity of the LF matrix. However, most existing NLF models have not adequately considered various LFs in different conditions, which might lead to negative impacts on the model performance. In order to address this issue, a novel NLF model, i.e., efficient NLF model (ENLF), is put forward to adequately reflect the various influences of LFs to the target matrix, thereby more freedom is introduced to find the solution of the established minimization problem. Furthermore, to alleviate the computational burden caused by the introduction of more latent factors, a so-called momentum-based additive gradient descent (MAGD) algorithm is employed to learn the model, where the truncating strategy is utilized during the update so as to guarantee the non-negativity of LFs. Finally, the empirical results on six real industrial data sets show that the ENLF model based on MAGD can achieve a desirable performance with relatively low time consumption.

Non-Negative Latent Factor Model Based on β-Divergence for Recommender Systems

Non-negative latent factor (NLF) models well represent high-dimensional and sparse (HiDS) matrices filled with non-negative data, which are frequently encountered in industrial applications like recommender systems. However, current NLF models mostly adopt Euclidean distance in their objective function, which represents a special case of a β-divergence function. Hence, it is highly desired to design a β-divergence-based NLF ( β-NLF) model that uses a β-divergence function, and investigate its performance in recommender systems as β varies. To do so, we first model β-NLF's learning objective with a β-divergence function. Subsequently, we deduce a general single latent factor-dependent, non-negative and multiplicative update scheme for β-NLF, and then design an efficient β-NLF algorithm. The experimental results on HiDS matrices from industrial applications indicate that by carefully choosing the value of β, β-NLF outperforms an NLF model with Euclidean distance in terms of accuracy for missing data prediction without increasing computational time. The research outcomes show the necessity of using an optimal β-divergence function in order to achieve the best performance of an NLF model on HiDS matrices. Hence, the proposed model has both theoretical and application significance.

Convergence Analysis of Single Latent Factor-Dependent, Nonnegative, and Multiplicative Update-Based Nonnegative Latent Factor Models.

A single latent factor (LF)-dependent, nonnegative, and multiplicative update (SLF-NMU) learning algorithm is highly efficient in building a nonnegative LF (NLF) model defined on a high-dimensional and sparse (HiDS) matrix. However, convergence characteristics of such NLF models are never justified in theory. To address this issue, this study conducts rigorous convergence analysis for an SLF-NMU-based NLF model. The main idea is twofold: 1) proving that its learning objective keeps nonincreasing with its SLF-NMU-based learning rules via constructing specific auxiliary functions; and 2) proving that it converges to a stable equilibrium point with its SLF-NMU-based learning rules via analyzing the Karush-Kuhn-Tucker (KKT) conditions of its learning objective. Experimental results on ten HiDS matrices from real applications provide numerical evidence that indicates the correctness of the achieved proof.

A Fast Non-Negative Latent Factor Model Based on Generalized Momentum Method

Non-negative latent factor (NLF) models can efficiently acquire useful knowledge from high-dimensional and sparse (HiDS) matrices filled with non-negative data. Single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) is an efficient algorithm for building an NLF model on an HiDS matrix, yet it suffers slow convergence. A momentum method is frequently adopted to accelerate a learning algorithm, but it is incompatible with those implicitly adopting gradients like SLF-NMU. To build a fast NLF (FNLF) model, we propose a generalized momentum method compatible with SLF-NMU. With it, we further propose a single latent factor-dependent non-negative, multiplicative and momentum-incorporated update algorithm, thereby achieving an FNLF model. Empirical studies on six HiDS matrices from industrial application indicate that an FNLF model outperforms an NLF model in terms of both convergence rate and prediction accuracy for missing data. Hence, compared with an NLF model, an FNLF model is more practical in industrial applications.

Momentum-Incorporated Symmetric Non-Negative Latent Factor Models

Symmetric high-dimensional and sparse (SHiDS) networks are frequently found in various industrial applications. A symmetric non-negative latent factor (SNLF) model can acquire essential features from them precisely, yet it suffers from slow convergence. To address this issue, this article integrates a generalized momentum method into a symmetric, single latent factor-dependent, non-negative and multiplicative update (S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> LF-NMU) algorithm, thereby achieving a <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</u> omentum-incorporated, <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</u> ymmetric, <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</u> ingle-latent-factor-dependent <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</u> on-negative-multiplicative-update (MS <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> N) algorithm. Based on an MS <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> N algorithm, momentum-incorporated symmetric non-negative latent factor (MSNLF) models are proposed for an SHiDS network, which ensures fast convergence as well as high representative learning ability. Empirical studies on four SHiDS networks from industrial applications demonstrate that compared with state-of-the-art models, the proposed MSNLF models have significantly higher computational efficiency and representative learning ability.

An Inherently Nonnegative Latent Factor Model for High-Dimensional and Sparse Matrices from Industrial Applications

High-dimensional and sparse (HiDS) matrices are commonly encountered in many big-data-related and industrial applications like recommender systems. When acquiring useful patterns from them, nonnegative matrix factorization (NMF) models have proven to be highly effective owing to their fine representativeness of the nonnegative data. However, current NMF techniques suffer from: 1) inefficiency in addressing HiDS matrices; and 2) constraints in their training schemes. To address these issues, this paper proposes to extract nonnegative latent factors (NLFs) from HiDS matrices via a novel inherently NLF (INLF) model. It bridges the output factors and decision variables via a single-element-dependent mapping function, thereby making the parameter training unconstrained and compatible with general training schemes on the premise of maintaining the nonnegativity constraints. Experimental results on six HiDS matrices arising from industrial applications indicate that INLF is able to acquire NLFs from them more efficiently than any existing method does.

Symmetric and Nonnegative Latent Factor Models for Undirected, High-Dimensional, and Sparse Networks in Industrial Applications

Undirected, high-dimensional, and sparse (HiDS) networks are frequently encountered in industrial applications. They contain rich knowledge regarding various useful patterns. Nonnegative latent factor (NLF) models are effective and efficient in extracting useful knowledge from directed networks. However, they cannot describe the symmetry of an undirected network. For addressing this issue, this paper analyzes the extraction process of NLFs on asymmetric and symmetric matrices, respectively, thereby innovatively achieving the symmetric and nonnegative latent factor (SNLF) models for undirected, HiDS networks. The proposed SNLF models are equipped with: 1) high efficiency; 2) nonnegativity; and 3) symmetry. Experimental results on real networks show that the SNLF models are able to: 1) describe the symmetry of the target network rigorously; 2) ensure the nonnegativity of resultant latent factors; and 3) achieve high computational efficiency when addressing data analysis tasks like missing data estimation.

A Novel Approach to Extracting Non-Negative Latent Factors From Non-Negative Big Sparse Matrices

An inherently non-negative latent factor model is proposed to extract non-negative latent factors from non-negative big sparse matrices efficiently and effectively. A single-element-dependent sigmoid function connects output latent factors with decision variables, such that non-negativity constraints on the output latent factors are always fulfilled and thus successfully separated from the training process with respect to the decision variables. Consequently, the proposed model can be easily and fast built with excellent prediction accuracy. Experimental results on an industrial size sparse matrix are given to verify its outstanding performance and suitability for industrial applications.

Generating Highly Accurate Predictions for Missing QoS Data via Aggregating Nonnegative Latent Factor Models.

Automatic Web-service selection is an important research topic in the domain of service computing. During this process, reliable predictions for quality of service (QoS) based on historical service invocations are vital to users. This work aims at making highly accurate predictions for missing QoS data via building an ensemble of nonnegative latent factor (NLF) models. Its motivations are: 1) the fulfillment of nonnegativity constraints can better represent the positive value nature of QoS data, thereby boosting the prediction accuracy and 2) since QoS prediction is a learning task, it is promising to further improve the prediction accuracy with a carefully designed ensemble model. To achieve this, we first implement an NLF model for QoS prediction. This model is then diversified through feature sampling and randomness injection to form a diversified NLF model, based on which an ensemble is built. Comparison results between the proposed ensemble and several widely employed and state-of-the-art QoS predictors on two large, real data sets demonstrate that the former can outperform the latter well in terms of prediction accuracy.