Matrix Factorization Framework Research Articles

Leveraging user’s preference and social circle for personalized recommendation via matrix factorization with sub-linear convergence rate

Recommender systems aim to filter information effectively and recommend useful sources to match users’ requirements. However, the exponential growth of information in recent social networks may cause low prediction accuracy for recommendation systems. This article proposes a unified personalized recommendation architecture referred to as PSRec, which incorporates user preference and social relationship into matrix factorization framework. Specifically, PSRec generates two collections for textual reviews and contextual information respectively, and performs preference learning for each user via the Latent Dirichlet Allocation topic model. Moreover, PSRec exploits the inner relations within the social circle for recommendation, including direct trust relationship and indirect trust relationship. Additionally, it’s certificated that PSRec can converge at a sub-linear rate via theoretical analysis. Experimental results over DoubanMovie, CiaoDVDs and Yelp demonstrate the superiority of the proposed PSRec, which can achieve significant improvements and provide much better user experience while compared with other benchmark models.

Temporal Autoregressive Matrix Factorization for High-Dimensional Time Series Prediction of OSS.

Open-source software (OSS) plays an increasingly significant role in modern software development tendency, so accurate prediction of the future development of OSS has become an essential topic. The behavioral data of different open-source software are closely related to their development prospects. However, most of these behavioral data are typical high-dimensional time series data streams with noise and missing values. Hence, accurate prediction on such cluttered data requires the model to be highly scalable, which is not a property of traditional time series prediction models. To this end, we propose a temporal autoregressive matrix factorization (TAMF) framework that supports data-driven temporal learning and prediction. Specifically, we first construct a trend and period autoregressive model to extract trend and period features from OSS behavioral data, and then combine the regression model with a graph-based matrix factorization (MF) to complete the missing values by exploiting the correlations among the time series data. Finally, use the trained regression model to make predictions on the target data. This scheme ensures that TAMF can be applied to different types of high-dimensional time series data and thus has high versatility. We selected ten real developer behavior data from GitHub for case analysis. The experimental results show that TAMF has good scalability and prediction accuracy.

Multi-view clustering based on pairwise co-regularization and robust dual graph non-negative matrix factorization

Multi-view clustering has been a hotspot in the field of machine learning and pattern recognition, and methods based on non-negative matrix factorization have gained attention for their simplicity and interpretability. Despite these methods achieving great clustering performance, there may also be some limitations, such as the full structural information of data and the similarity between different views not being considered. This paper proposes a novel multi-view clustering algorithm based on pairwise co-regularization and robust dual graph non-negative matrix factorization. Firstly, the l2,p-norm is applied in non-negative matrix factorization framework to enhance model robustness. Next, pairwise co-regularization is utilized to extract the inter-view information of all views. Then, graph dual regularization is applied to preserve the structure information of the data and feature spaces. Lastly, an auto-weighted strategy is introduced to assign appropriate weights to each view. In addition, an iterative updating optimization scheme for the proposed algorithm is developed, and the convergence proof of the scheme is provided. The experimental results on twelve real-world datasets show that the proposed algorithm is both effective and efficient, compared with three classical and six state-of-the-art algorithms.

Forecasting Urban Traffic States with Sparse Data Using Hankel Temporal Matrix Factorization

Forecasting urban traffic states is crucial to transportation network monitoring and management, playing an important role in the decision-making process. Despite the substantial progress that has been made in developing accurate, efficient, and reliable algorithms for traffic forecasting, most existing approaches fail to handle sparsity, high-dimensionality, and nonstationarity in traffic time series and seldom consider the temporal dependence between traffic states. To address these issues, this work presents a Hankel temporal matrix factorization (HTMF) model using the Hankel matrix in the lower dimensional spaces under a matrix factorization framework. In particular, we consider an alternating minimization scheme to optimize the factor matrices in matrix factorization and the Hankel matrix in the lower dimensional spaces simultaneously. To perform traffic state forecasting, we introduce two efficient estimation processes on real-time incremental data, including an online imputation (i.e., reconstruct missing values) and an online forecasting (i.e., estimate future data points). Through extensive experiments on the real-world Uber movement speed data set in Seattle, Washington, we empirically demonstrate the superior forecasting performance of HTMF over several baseline models and highlight the advantages of HTMF for addressing sparsity, nonstationarity, and short time series. History: Accepted by Ram Ramesh, Area Editor for Data Science & Machine Learning. Funding: This research was supported by the Institute for Data Valorisation, the Interuniversity Research Centre on Enterprise Networks, Logistics and Transportation, the National Natural Science Foundation of China [Grants 12371456, 72101049, 72232001], the Sichuan Science and Technology Program [Grant 2024NSFJQ0038], and the Fundamental Research Funds for the Central Universities [Grant DUT23RC(3)045].

Quadratic Matrix Factorization With Applications to Manifold Learning.

Matrix factorization is a popular framework for modeling low-rank data matrices. Motivated by manifold learning problems, this paper proposes a quadratic matrix factorization (QMF) framework to learn the curved manifold on which the dataset lies. Unlike local linear methods such as the local principal component analysis, QMF can better exploit the curved structure of the underlying manifold. Algorithmically, we propose an alternating minimization algorithm to optimize QMF and establish its theoretical convergence properties. To avoid possible over-fitting, we then propose a regularized QMF algorithm and discuss how to tune its regularization parameter. Finally, we elaborate how to apply the regularized QMF to manifold learning problems. Experiments on a synthetic manifold learning dataset and three real-world datasets, including the MNIST handwritten dataset, a cryogenic electron microscopy dataset, and the Frey Face dataset, demonstrate the superiority of the proposed method over its competitors.

An ordered subsets orthogonal nonnegative matrix factorization framework with application to image clustering

An ordered subsets orthogonal nonnegative matrix factorization framework with application to image clustering

Robust hyperspectral unmixing using weighted group sparsity and minimum volume regularization

ABSTRACT Hyperspectral unmixing (HU) has been a vital technique in hyperspectral image processing. Over the past years, many algorithms have been developed based on the nonnegative matrix factorization (NMF) framework. However, the conventional methods based on the NMF need prior information of the number of endmembers, which is usually estimated by a separate procedure. Therefore, the precision of estimating the number of endmembers will impact the subsequent unmixing effect. Thus, collaborating between the two processes is desired to avoid the propagation of errors. This paper proposes a new NMF-based method, which can estimate the number of endmembers and extract the endmembers and the abundance maps. Firstly, we introduce the weighted group sparsity regularization to adaptively eliminate the abundances corresponding to the redundant endmembers, while the abundances of the true endmembers are preserved. Then the number of endmembers can be determined by the reconstruction error function and the l 2 -norm of the abundance vectors. Secondly, since the simplex determined by the endmembers has the minimum volume among all simplexes that circumscribe the data points, we impose the total variational regularization to force the endmembers to be close to each other to minimize the simplex volume. And because the total variational regularization avoids the use of noisy pixels, it can decrease the influence of noise and improve the robustness of the solution. For the proposed model, we design an effective algorithm to solve it. Also, the experimental results based on the synthetic data and real data demonstrate that the proposed method can identify the endmembers number well and achieve robust unmixing results.

Robust and Effective: A Deep Matrix Factorization Framework for Classification.

For complex data, high dimension and high noise are challenging problems, and deep matrix factorization shows great potential in data dimensionality reduction. In this article, a novel robust and effective deep matrix factorization framework is proposed. This method constructs a dual-angle feature for single-modal gene data to improve the effectiveness and robustness, which can solve the problem of high-dimensional tumor classification. The proposed framework consists of three parts, deep matrix factorization, double-angle decomposition, and feature purification. First, a robust deep matrix factorization (RDMF) model is proposed in the feature learning, to enhance the classification stability and obtain better feature when faced with noisy data. Second, a double-angle feature (RDMF-DA) is designed by cascading the RDMF features with sparse features, which contains the more comprehensive information in gene data. Third, to avoid the influence of redundant genes on the representation ability, a gene selection method is proposed to purify the features by RDMF-DA, based on the principle of sparse representation (SR) and gene coexpression. Finally, the proposed algorithm is applied to the gene expression profiling datasets, and the performance of the algorithm is fully verified.

Review of Patient Gene Profiles Obtained through a Non-Negative Matrix Factorization-Based Framework to Determine the Role Inflammation Plays in Neuroblastoma Pathogenesis.

Neuroblastoma is the most common extracranial solid tumor in children. It is a highly heterogeneous tumor consisting of different subcellular types and genetic abnormalities. Literature data confirm the biological and clinical complexity of this cancer, which requires a wider availability of gene targets for the implementation of personalized therapy. This paper presents a study of neuroblastoma samples from primary tumors of untreated patients. The focus of this analysis is to evaluate the impact that the inflammatory process may have on the pathogenesis of neuroblastoma. Eighty-eight gene profiles were selected and analyzed using a non-negative matrix factorization framework to extract a subset of genes relevant to the identification of an inflammatory phenotype, whose targets (PIK3CG, NFATC2, PIK3R2, VAV1, RAC2, COL6A2, COL6A3, COL12A1, COL14A1, ITGAL, ITGB7, FOS, PTGS2, PTPRC, ITPR3) allow further investigation. Based on the genetic signals automatically derived from the data used, neuroblastoma could be classified according to stage rather than as a "cold" or "poorly immunogenic" tumor.

Abstract 3494: KINOMO: A non-negative matrix factorization framework for recovering intra- and inter-tumoral heterogeneity from single-cell RNA-seq data

Abstract Single-cell RNA-sequencing (scRNA-seq) is a powerful technology to uncover cellular heterogeneity in tumor ecosystems. Due to differences in underlying gene load, direct comparison between patient samples is challenging, and this is further complicated by the sparsity of data matrices in scRNA-seq. Here, we present a factorization method called KINOMO (Kernel dIfferentiability correlation-based NOn- negative Matrix factorization algorithm using Kullback-Leibler divergence loss Optimization). This tool uses quadratic approximation approach for error correction and an iterative multiplicative approach, which improves the quality assessment of NMF-identified factorization, while mitigating biases introduced by inter-patient genomic variability. We benchmarked this new approach against nine different methods across 150 scRNA-seq experiments and find that KINOMO outperforms prior methods when evaluated with an adjusted Rand index (ARI), ranging 0.82-0.91 compared to 0.68-0.77. Thus, KINOMO provides an improved approach for determining coherent transcriptional programs (and meta-programs) from scRNA- seq data of cancer tissues, enabling comparison of patients with variable genomic backgrounds. Citation Format: Somnath Tagore, Yiping Wang, Jana Biermann, Edridge K. D'Souza, Karan D. Luthria, Raul Rabadan, Elham Azizi, Benjamin Izar. KINOMO: A non-negative matrix factorization framework for recovering intra- and inter-tumoral heterogeneity from single-cell RNA-seq data [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 3494.

GCNSLIM: Graph convolutional network with sparse linear methods for e-government service recommendation

E-government service recommendation aims to find target service items for users based on their preferences, behaviors and other information, which is one of the key technologies for information overload in e-government cloud platforms. However, it has not received adequate attention in comparison to other recommendation scenarios like news and music recommendation. Graph Convolutional Networks (GCNs) have made significant strides in Collaborative Filtering (CF) recommendations. However, existing GCN-based CF methods are mainly based on matrix factorization (MF) and incorporate some optimization techniques (e.g., contrastive learning) to enhance performance, which are not enough to handle the complexities of diverse real-world recommendation scenarios. We empirically find that when existing GCN-based CF methods are directly applied to e-government service recommendation, they are limited by the MF framework and showing poor performance. This is attributed to the fact that the equal treatment of users and items by MF is not appropriate for scenarios where the number of users and items is unbalanced. In this work, we propose a new model, GCNSLIM, which combines GCN and sparse linear methods (SLIM) instead of combining GCN and MF to accommodate e-government service recommendation. In particular, GCNSLIM explicitly injects high-order collaborative signals obtained from multi-layer light graph convolutions into the item similarity matrix in the SLIM framework, effectively improving the recommendation accuracy. In addition, we propose two optimization measures, removing layer 0 embedding and adding nonlinear activation, to further adapt to the characteristics of e-government service recommendation scenarios. Furthermore, we propose a joint optimization mode to enhance the adaptability for more diverse recommendation scenarios. We conduct extensive experiments on a real e-government service dataset and two common public datasets and demonstrate the effectiveness of GCNSLIM in recommendation accuracy and operational performance. Our implementation is available at https://github.com/songkk5/GCNSLIM.

MFAI: A Scalable Bayesian Matrix Factorization Approach to Leveraging Auxiliary Information

In various practical situations, matrix factorization methods suffer from poor data quality, such as high data sparsity and low signal-to-noise ratio (SNR). Here, we consider a matrix factorization problem by utilizing auxiliary information, which is massively available in real-world applications, to overcome the challenges caused by poor data quality. Unlike existing methods that mainly rely on simple linear models to combine auxiliary information with the main data matrix, we propose to integrate gradient boosted trees in the probabilistic matrix factorization framework to effectively leverage auxiliary information (MFAI). Thus, MFAI naturally inherits several salient features of gradient boosted trees, such as the capability of flexibly modeling nonlinear relationships and robustness to irrelevant features and missing values in auxiliary information. The parameters in MFAI can be automatically determined under the empirical Bayes framework, making it adaptive to the utilization of auxiliary information and immune to overfitting. Moreover, MFAI is computationally efficient and scalable to large datasets by exploiting variational inference. We demonstrate the advantages of MFAI through comprehensive numerical results from simulation studies and real data analyses. Our approach is implemented in the R package mfair available at https://github.com/YangLabHKUST/mfair.

Inter- and intra-hypergraph regularized nonnegative matrix factorization with hybrid constraints

The accurate low rank representation of high-dimensional data learned by the manifold regularized nonnegative matrix factorization framework is effective in data clustering. In previous work, this work has mainly been solved in the way of similarity matrix induction. To further increase the efficacy of low-rank representations, we propose a novel semi-supervised non-negative matrix factorization (NMF) model in this study called inter- and intra-hypergraph regularized non-negative matrix factorization with hybrid constraints (IGNMFC). IGNMFC constructs intra-hypergraph regularization and intra-hypergraph regularization by hypergraph learning, which can precisely induce high-dimensional data to map toward low-dimensional. Moreover, hybrid constraints are introduced to improve the exclusivity and sparsity of low-dimensional representations, and the result accounts for the benefit of this method in learning distinguishable subspace representations. Finally, IGNMFC is transformed into an optimal problem and an efficient iteration rule is proposed. Experiments on several datasets demonstrate that the proposed method outperforms the state-of-the-art NMF algorithms, and can achieve at least 7.9%∼15% accuracy improvement in most cases.

Pure kernel graph fusion tensor subspace clustering under non-negative matrix factorization framework

Currently, graph-based multi-kernel subspace clustering methods have achieved rich research results in dealing with nonlinear data structures. However, most of the methods still have the following two limitations: (1) the model optimization goal is finding the optimal consensus kernel rather than the optimal affinity graph, which leads to kernel noise always interfering with the learning of the target affinity graph in the optimization process; (2) in the process of finding the optimal affinity graph, only the correlation between different kernel views is exploited, ignoring the higher-order correlation between multiple kernel views. In light of this, we propose a pure kernel graph fusion tensor (PKGT) subspace clustering under non-negative matrix factorization (NMF) framework. Specifically, we first use NMF to construct local affinity feature graphs with standard block diagonal structure for each base kernel matrix, and then achieve adaptive weight assignment by optimizing the inconsistencies among multiple local affinity feature graphs. Finally, we use tensor to capture the higher-order correlation between multiple local affinity feature graphs to provide more feature information for the learning of optimal target affinity graph, which enhances the model clustering performance. Extensive experiments on nine real datasets demonstrate that PKGT has superior clustering performance. The code is available at https://github.com/ZS0124/PKGT.

UMA-MF: A Unified Multi-CPU/GPU Asynchronous Computing Framework for SGD-Based Matrix Factorization

UMA-MF: A Unified Multi-CPU/GPU Asynchronous Computing Framework for SGD-Based Matrix Factorization

Entropy Minimizing Matrix Factorization.

Nonnegative matrix factorization (NMF) is a widely used data analysis technique and has yielded impressive results in many real-world tasks. Generally, existing NMF methods represent each sample with several centroids and find the optimal centroids by minimizing the sum of the residual errors. However, outliers deviating from the normal data distribution may have large residues and then dominate the objective value. In this study, an entropy minimizing matrix factorization (EMMF) framework is developed to tackle the above problem. Considering that outliers are usually much less than the normal samples, a new entropy loss function is established for matrix factorization, which minimizes the entropy of the residue distribution and allows a few samples to have large errors. In this way, the outliers do not affect the approximation of normal samples. Multiplicative updating rules for EMMF are derived, and the convergence is proven theoretically. In addition, a Graph regularized version of EMMF (G-EMMF) is also presented, which uses a data graph to capture the data relationship. Clustering results on various synthetic and real-world datasets demonstrate the advantages of the proposed models, and the effectiveness is also verified through the comparison with state-of-the-art methods.

Back temporal autoregressive matrix factorization for high-dimensional time series prediction

While prediction for large-scale multidimensional spatio-temporal data is widely used in applications such as business analytics and open source metrics, service data collected from these applications often have values that are missing from the data. The accuracy of prediction results requires that the prediction model used is highly robust and scalable, something that is typically lacking in traditional prediction models. To this end, we propose a back temporal autoregressive matrix factorization (BTAMF) framework, which can construct an efficient prediction model for time series data with missing values by mining the trend and periodic characteristics of the prediction results. It first uses the target data and its left neighbors to construct trend autoregressive and then uses the target data and its associated periodic data to construct periodic autoregressive to mine the trend and periodicity properties of time series data. We then integrate the tensor/matrix factorization and both autoregressive models into a single optimization model for missing value completion and prediction through optimization of the objective functions. This scheme, which exploits global and local information of the target data, can directly predict the data without pre-completing the missing values, which provides a strong guarantee of the temporal integrity of the time series data. To validate the efficient completion and prediction effects of BTAMF in real-world scenarios, we select ten real-world behavioral datasets from GitHub and seven comparison models for experiments. Our experimental results show that BTAMF has a stronger advantage over current state-of-the-art methods in terms of scalability and prediction accuracy.

How heterogeneous social influence acts on human decision-making in online social networks

Social network-based tasks frequently emphasize the influence of direct neighbors on human decision-making. However, there is no objective consensus on whether direct neighbors are influential in all circumstances, or any other type of “neighbors” who have a greater impact. In this study, we investigate the patterns of heterogeneous social influence on human decision-making from the perspectives of opinions, behaviors, preferences and decision probabilities across three large-scale online networks. Our results demonstrate that the influence of similar behavior during exploration on decision-making outperforms that of explicit social relationships regardless of relationship sparsity, yet is associated with shared experiences, network structures, and individual attributes. Specifically, at low levels of experiences, similar behavior during exploration drives human behavioral shifts from bystanders to participants more effectively than explicit social relationships. Further incorporating heterogeneous influence into opinion dynamics through the matrix factorization framework, we reveal the mechanism of heterogeneous social influence acting on decision-making in online networks, with implications for understanding human decision-making.

Price-aware matrix factorization model for personalized recommendations

Recommendation methods aim to assist consumers in their decision-making process to find products that they are quite likely interested in. Current recommendation methods generally use online consumer reviews or ratings to predict consumers’ preferences for products. In e-commerce transactions, price plays a significant role in consumers’ purchase decisions. And each consumer's preference for product prices is specific. In this paper, we propose a novel price-aware recommendation method based on the matrix factorization model which fully considers the effect of price. We distill the price preferences of consumers from ratings and discover consumers’ real preferences for products. We further calculate the price sensitivities of consumers, not only considering the difference in price preference between a consumer and others but also focusing on the consumers’ price preferences in a specific price range. The final predicted ratings are calculated by adding the term of price effect into the matrix factorization framework. The results show that the proposed method has achieved high accuracy and performed better than several existing methods in both rating prediction and Top-N recommendation. And the products we recommend for consumers are more in line with their price preferences. Moreover, we find that, to some extent, the proposed method can solve the long-tail product recommendation problem with the consideration of the price effect.

Graph Regularized Probabilistic Matrix Factorization for Drug-Drug Interactions Prediction.

Co-administration of two or more drugs simultaneously can result in adverse drug reactions. Identifying drug-drug interactions (DDIs) is necessary, especially for drug development and for repurposing old drugs. DDI prediction can be viewed as a matrix completion task, for which matrix factorization (MF) appears as a suitable solution. This paper presents a novel Graph Regularized Probabilistic Matrix Factorization (GRPMF) method, which incorporates expert knowledge through a novel graph-based regularization strategy within an MF framework. An efficient and sounded optimization algorithm is proposed to solve the resulting non-convex problem in an alternating fashion. The performance of the proposed method is evaluated through the DrugBank dataset, and comparisons are provided against state-of-the-art techniques. The results demonstrate the superior performance of GRPMF when compared to its counterparts.