Matrix Factorization Framework Research Articles

Forecasting Urban Traffic States with Sparse Data Using Hankel Temporal Matrix Factorization

Forecasting urban traffic states is crucial to transportation network monitoring and management, playing an important role in the decision-making process. Despite the substantial progress that has been made in developing accurate, efficient, and reliable algorithms for traffic forecasting, most existing approaches fail to handle sparsity, high-dimensionality, and nonstationarity in traffic time series and seldom consider the temporal dependence between traffic states. To address these issues, this work presents a Hankel temporal matrix factorization (HTMF) model using the Hankel matrix in the lower dimensional spaces under a matrix factorization framework. In particular, we consider an alternating minimization scheme to optimize the factor matrices in matrix factorization and the Hankel matrix in the lower dimensional spaces simultaneously. To perform traffic state forecasting, we introduce two efficient estimation processes on real-time incremental data, including an online imputation (i.e., reconstruct missing values) and an online forecasting (i.e., estimate future data points). Through extensive experiments on the real-world Uber movement speed data set in Seattle, Washington, we empirically demonstrate the superior forecasting performance of HTMF over several baseline models and highlight the advantages of HTMF for addressing sparsity, nonstationarity, and short time series. History: Accepted by Ram Ramesh, Area Editor for Data Science & Machine Learning. Funding: This research was supported by the Institute for Data Valorisation, the Interuniversity Research Centre on Enterprise Networks, Logistics and Transportation, the National Natural Science Foundation of China [Grants 12371456, 72101049, 72232001], the Sichuan Science and Technology Program [Grant 2024NSFJQ0038], and the Fundamental Research Funds for the Central Universities [Grant DUT23RC(3)045].

Quadratic Matrix Factorization With Applications to Manifold Learning.

Matrix factorization is a popular framework for modeling low-rank data matrices. Motivated by manifold learning problems, this paper proposes a quadratic matrix factorization (QMF) framework to learn the curved manifold on which the dataset lies. Unlike local linear methods such as the local principal component analysis, QMF can better exploit the curved structure of the underlying manifold. Algorithmically, we propose an alternating minimization algorithm to optimize QMF and establish its theoretical convergence properties. To avoid possible over-fitting, we then propose a regularized QMF algorithm and discuss how to tune its regularization parameter. Finally, we elaborate how to apply the regularized QMF to manifold learning problems. Experiments on a synthetic manifold learning dataset and three real-world datasets, including the MNIST handwritten dataset, a cryogenic electron microscopy dataset, and the Frey Face dataset, demonstrate the superiority of the proposed method over its competitors.

An ordered subsets orthogonal nonnegative matrix factorization framework with application to image clustering

An ordered subsets orthogonal nonnegative matrix factorization framework with application to image clustering

Robust hyperspectral unmixing using weighted group sparsity and minimum volume regularization

ABSTRACT Hyperspectral unmixing (HU) has been a vital technique in hyperspectral image processing. Over the past years, many algorithms have been developed based on the nonnegative matrix factorization (NMF) framework. However, the conventional methods based on the NMF need prior information of the number of endmembers, which is usually estimated by a separate procedure. Therefore, the precision of estimating the number of endmembers will impact the subsequent unmixing effect. Thus, collaborating between the two processes is desired to avoid the propagation of errors. This paper proposes a new NMF-based method, which can estimate the number of endmembers and extract the endmembers and the abundance maps. Firstly, we introduce the weighted group sparsity regularization to adaptively eliminate the abundances corresponding to the redundant endmembers, while the abundances of the true endmembers are preserved. Then the number of endmembers can be determined by the reconstruction error function and the l 2 -norm of the abundance vectors. Secondly, since the simplex determined by the endmembers has the minimum volume among all simplexes that circumscribe the data points, we impose the total variational regularization to force the endmembers to be close to each other to minimize the simplex volume. And because the total variational regularization avoids the use of noisy pixels, it can decrease the influence of noise and improve the robustness of the solution. For the proposed model, we design an effective algorithm to solve it. Also, the experimental results based on the synthetic data and real data demonstrate that the proposed method can identify the endmembers number well and achieve robust unmixing results.

Robust and Effective: A Deep Matrix Factorization Framework for Classification.

For complex data, high dimension and high noise are challenging problems, and deep matrix factorization shows great potential in data dimensionality reduction. In this article, a novel robust and effective deep matrix factorization framework is proposed. This method constructs a dual-angle feature for single-modal gene data to improve the effectiveness and robustness, which can solve the problem of high-dimensional tumor classification. The proposed framework consists of three parts, deep matrix factorization, double-angle decomposition, and feature purification. First, a robust deep matrix factorization (RDMF) model is proposed in the feature learning, to enhance the classification stability and obtain better feature when faced with noisy data. Second, a double-angle feature (RDMF-DA) is designed by cascading the RDMF features with sparse features, which contains the more comprehensive information in gene data. Third, to avoid the influence of redundant genes on the representation ability, a gene selection method is proposed to purify the features by RDMF-DA, based on the principle of sparse representation (SR) and gene coexpression. Finally, the proposed algorithm is applied to the gene expression profiling datasets, and the performance of the algorithm is fully verified.

Review of Patient Gene Profiles Obtained through a Non-Negative Matrix Factorization-Based Framework to Determine the Role Inflammation Plays in Neuroblastoma Pathogenesis.

Neuroblastoma is the most common extracranial solid tumor in children. It is a highly heterogeneous tumor consisting of different subcellular types and genetic abnormalities. Literature data confirm the biological and clinical complexity of this cancer, which requires a wider availability of gene targets for the implementation of personalized therapy. This paper presents a study of neuroblastoma samples from primary tumors of untreated patients. The focus of this analysis is to evaluate the impact that the inflammatory process may have on the pathogenesis of neuroblastoma. Eighty-eight gene profiles were selected and analyzed using a non-negative matrix factorization framework to extract a subset of genes relevant to the identification of an inflammatory phenotype, whose targets (PIK3CG, NFATC2, PIK3R2, VAV1, RAC2, COL6A2, COL6A3, COL12A1, COL14A1, ITGAL, ITGB7, FOS, PTGS2, PTPRC, ITPR3) allow further investigation. Based on the genetic signals automatically derived from the data used, neuroblastoma could be classified according to stage rather than as a "cold" or "poorly immunogenic" tumor.

Abstract 3494: KINOMO: A non-negative matrix factorization framework for recovering intra- and inter-tumoral heterogeneity from single-cell RNA-seq data

Abstract Single-cell RNA-sequencing (scRNA-seq) is a powerful technology to uncover cellular heterogeneity in tumor ecosystems. Due to differences in underlying gene load, direct comparison between patient samples is challenging, and this is further complicated by the sparsity of data matrices in scRNA-seq. Here, we present a factorization method called KINOMO (Kernel dIfferentiability correlation-based NOn- negative Matrix factorization algorithm using Kullback-Leibler divergence loss Optimization). This tool uses quadratic approximation approach for error correction and an iterative multiplicative approach, which improves the quality assessment of NMF-identified factorization, while mitigating biases introduced by inter-patient genomic variability. We benchmarked this new approach against nine different methods across 150 scRNA-seq experiments and find that KINOMO outperforms prior methods when evaluated with an adjusted Rand index (ARI), ranging 0.82-0.91 compared to 0.68-0.77. Thus, KINOMO provides an improved approach for determining coherent transcriptional programs (and meta-programs) from scRNA- seq data of cancer tissues, enabling comparison of patients with variable genomic backgrounds. Citation Format: Somnath Tagore, Yiping Wang, Jana Biermann, Edridge K. D'Souza, Karan D. Luthria, Raul Rabadan, Elham Azizi, Benjamin Izar. KINOMO: A non-negative matrix factorization framework for recovering intra- and inter-tumoral heterogeneity from single-cell RNA-seq data [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 3494.

GCNSLIM: Graph convolutional network with sparse linear methods for e-government service recommendation

E-government service recommendation aims to find target service items for users based on their preferences, behaviors and other information, which is one of the key technologies for information overload in e-government cloud platforms. However, it has not received adequate attention in comparison to other recommendation scenarios like news and music recommendation. Graph Convolutional Networks (GCNs) have made significant strides in Collaborative Filtering (CF) recommendations. However, existing GCN-based CF methods are mainly based on matrix factorization (MF) and incorporate some optimization techniques (e.g., contrastive learning) to enhance performance, which are not enough to handle the complexities of diverse real-world recommendation scenarios. We empirically find that when existing GCN-based CF methods are directly applied to e-government service recommendation, they are limited by the MF framework and showing poor performance. This is attributed to the fact that the equal treatment of users and items by MF is not appropriate for scenarios where the number of users and items is unbalanced. In this work, we propose a new model, GCNSLIM, which combines GCN and sparse linear methods (SLIM) instead of combining GCN and MF to accommodate e-government service recommendation. In particular, GCNSLIM explicitly injects high-order collaborative signals obtained from multi-layer light graph convolutions into the item similarity matrix in the SLIM framework, effectively improving the recommendation accuracy. In addition, we propose two optimization measures, removing layer 0 embedding and adding nonlinear activation, to further adapt to the characteristics of e-government service recommendation scenarios. Furthermore, we propose a joint optimization mode to enhance the adaptability for more diverse recommendation scenarios. We conduct extensive experiments on a real e-government service dataset and two common public datasets and demonstrate the effectiveness of GCNSLIM in recommendation accuracy and operational performance. Our implementation is available at https://github.com/songkk5/GCNSLIM.

MFAI: A Scalable Bayesian Matrix Factorization Approach to Leveraging Auxiliary Information

In various practical situations, matrix factorization methods suffer from poor data quality, such as high data sparsity and low signal-to-noise ratio (SNR). Here, we consider a matrix factorization problem by utilizing auxiliary information, which is massively available in real-world applications, to overcome the challenges caused by poor data quality. Unlike existing methods that mainly rely on simple linear models to combine auxiliary information with the main data matrix, we propose to integrate gradient boosted trees in the probabilistic matrix factorization framework to effectively leverage auxiliary information (MFAI). Thus, MFAI naturally inherits several salient features of gradient boosted trees, such as the capability of flexibly modeling nonlinear relationships and robustness to irrelevant features and missing values in auxiliary information. The parameters in MFAI can be automatically determined under the empirical Bayes framework, making it adaptive to the utilization of auxiliary information and immune to overfitting. Moreover, MFAI is computationally efficient and scalable to large datasets by exploiting variational inference. We demonstrate the advantages of MFAI through comprehensive numerical results from simulation studies and real data analyses. Our approach is implemented in the R package mfair available at https://github.com/YangLabHKUST/mfair.

Temporal Autoregressive Matrix Factorization for High-Dimensional Time Series Prediction of OSS.

Open-source software (OSS) plays an increasingly significant role in modern software development tendency, so accurate prediction of the future development of OSS has become an essential topic. The behavioral data of different open-source software are closely related to their development prospects. However, most of these behavioral data are typical high-dimensional time series data streams with noise and missing values. Hence, accurate prediction on such cluttered data requires the model to be highly scalable, which is not a property of traditional time series prediction models. To this end, we propose a temporal autoregressive matrix factorization (TAMF) framework that supports data-driven temporal learning and prediction. Specifically, we first construct a trend and period autoregressive model to extract trend and period features from OSS behavioral data, and then combine the regression model with a graph-based matrix factorization (MF) to complete the missing values by exploiting the correlations among the time series data. Finally, use the trained regression model to make predictions on the target data. This scheme ensures that TAMF can be applied to different types of high-dimensional time series data and thus has high versatility. We selected ten real developer behavior data from GitHub for case analysis. The experimental results show that TAMF has good scalability and prediction accuracy.

Inter- and intra-hypergraph regularized nonnegative matrix factorization with hybrid constraints

The accurate low rank representation of high-dimensional data learned by the manifold regularized nonnegative matrix factorization framework is effective in data clustering. In previous work, this work has mainly been solved in the way of similarity matrix induction. To further increase the efficacy of low-rank representations, we propose a novel semi-supervised non-negative matrix factorization (NMF) model in this study called inter- and intra-hypergraph regularized non-negative matrix factorization with hybrid constraints (IGNMFC). IGNMFC constructs intra-hypergraph regularization and intra-hypergraph regularization by hypergraph learning, which can precisely induce high-dimensional data to map toward low-dimensional. Moreover, hybrid constraints are introduced to improve the exclusivity and sparsity of low-dimensional representations, and the result accounts for the benefit of this method in learning distinguishable subspace representations. Finally, IGNMFC is transformed into an optimal problem and an efficient iteration rule is proposed. Experiments on several datasets demonstrate that the proposed method outperforms the state-of-the-art NMF algorithms, and can achieve at least 7.9%∼15% accuracy improvement in most cases.

Pure kernel graph fusion tensor subspace clustering under non-negative matrix factorization framework

Currently, graph-based multi-kernel subspace clustering methods have achieved rich research results in dealing with nonlinear data structures. However, most of the methods still have the following two limitations: (1) the model optimization goal is finding the optimal consensus kernel rather than the optimal affinity graph, which leads to kernel noise always interfering with the learning of the target affinity graph in the optimization process; (2) in the process of finding the optimal affinity graph, only the correlation between different kernel views is exploited, ignoring the higher-order correlation between multiple kernel views. In light of this, we propose a pure kernel graph fusion tensor (PKGT) subspace clustering under non-negative matrix factorization (NMF) framework. Specifically, we first use NMF to construct local affinity feature graphs with standard block diagonal structure for each base kernel matrix, and then achieve adaptive weight assignment by optimizing the inconsistencies among multiple local affinity feature graphs. Finally, we use tensor to capture the higher-order correlation between multiple local affinity feature graphs to provide more feature information for the learning of optimal target affinity graph, which enhances the model clustering performance. Extensive experiments on nine real datasets demonstrate that PKGT has superior clustering performance. The code is available at https://github.com/ZS0124/PKGT.

UMA-MF: A Unified Multi-CPU/GPU Asynchronous Computing Framework for SGD-Based Matrix Factorization

UMA-MF: A Unified Multi-CPU/GPU Asynchronous Computing Framework for SGD-Based Matrix Factorization

Entropy Minimizing Matrix Factorization.

Nonnegative matrix factorization (NMF) is a widely used data analysis technique and has yielded impressive results in many real-world tasks. Generally, existing NMF methods represent each sample with several centroids and find the optimal centroids by minimizing the sum of the residual errors. However, outliers deviating from the normal data distribution may have large residues and then dominate the objective value. In this study, an entropy minimizing matrix factorization (EMMF) framework is developed to tackle the above problem. Considering that outliers are usually much less than the normal samples, a new entropy loss function is established for matrix factorization, which minimizes the entropy of the residue distribution and allows a few samples to have large errors. In this way, the outliers do not affect the approximation of normal samples. Multiplicative updating rules for EMMF are derived, and the convergence is proven theoretically. In addition, a Graph regularized version of EMMF (G-EMMF) is also presented, which uses a data graph to capture the data relationship. Clustering results on various synthetic and real-world datasets demonstrate the advantages of the proposed models, and the effectiveness is also verified through the comparison with state-of-the-art methods.

Back temporal autoregressive matrix factorization for high-dimensional time series prediction

While prediction for large-scale multidimensional spatio-temporal data is widely used in applications such as business analytics and open source metrics, service data collected from these applications often have values that are missing from the data. The accuracy of prediction results requires that the prediction model used is highly robust and scalable, something that is typically lacking in traditional prediction models. To this end, we propose a back temporal autoregressive matrix factorization (BTAMF) framework, which can construct an efficient prediction model for time series data with missing values by mining the trend and periodic characteristics of the prediction results. It first uses the target data and its left neighbors to construct trend autoregressive and then uses the target data and its associated periodic data to construct periodic autoregressive to mine the trend and periodicity properties of time series data. We then integrate the tensor/matrix factorization and both autoregressive models into a single optimization model for missing value completion and prediction through optimization of the objective functions. This scheme, which exploits global and local information of the target data, can directly predict the data without pre-completing the missing values, which provides a strong guarantee of the temporal integrity of the time series data. To validate the efficient completion and prediction effects of BTAMF in real-world scenarios, we select ten real-world behavioral datasets from GitHub and seven comparison models for experiments. Our experimental results show that BTAMF has a stronger advantage over current state-of-the-art methods in terms of scalability and prediction accuracy.

How heterogeneous social influence acts on human decision-making in online social networks

Social network-based tasks frequently emphasize the influence of direct neighbors on human decision-making. However, there is no objective consensus on whether direct neighbors are influential in all circumstances, or any other type of “neighbors” who have a greater impact. In this study, we investigate the patterns of heterogeneous social influence on human decision-making from the perspectives of opinions, behaviors, preferences and decision probabilities across three large-scale online networks. Our results demonstrate that the influence of similar behavior during exploration on decision-making outperforms that of explicit social relationships regardless of relationship sparsity, yet is associated with shared experiences, network structures, and individual attributes. Specifically, at low levels of experiences, similar behavior during exploration drives human behavioral shifts from bystanders to participants more effectively than explicit social relationships. Further incorporating heterogeneous influence into opinion dynamics through the matrix factorization framework, we reveal the mechanism of heterogeneous social influence acting on decision-making in online networks, with implications for understanding human decision-making.

Price-aware matrix factorization model for personalized recommendations

Recommendation methods aim to assist consumers in their decision-making process to find products that they are quite likely interested in. Current recommendation methods generally use online consumer reviews or ratings to predict consumers’ preferences for products. In e-commerce transactions, price plays a significant role in consumers’ purchase decisions. And each consumer's preference for product prices is specific. In this paper, we propose a novel price-aware recommendation method based on the matrix factorization model which fully considers the effect of price. We distill the price preferences of consumers from ratings and discover consumers’ real preferences for products. We further calculate the price sensitivities of consumers, not only considering the difference in price preference between a consumer and others but also focusing on the consumers’ price preferences in a specific price range. The final predicted ratings are calculated by adding the term of price effect into the matrix factorization framework. The results show that the proposed method has achieved high accuracy and performed better than several existing methods in both rating prediction and Top-N recommendation. And the products we recommend for consumers are more in line with their price preferences. Moreover, we find that, to some extent, the proposed method can solve the long-tail product recommendation problem with the consideration of the price effect.

Graph Regularized Probabilistic Matrix Factorization for Drug-Drug Interactions Prediction.

Co-administration of two or more drugs simultaneously can result in adverse drug reactions. Identifying drug-drug interactions (DDIs) is necessary, especially for drug development and for repurposing old drugs. DDI prediction can be viewed as a matrix completion task, for which matrix factorization (MF) appears as a suitable solution. This paper presents a novel Graph Regularized Probabilistic Matrix Factorization (GRPMF) method, which incorporates expert knowledge through a novel graph-based regularization strategy within an MF framework. An efficient and sounded optimization algorithm is proposed to solve the resulting non-convex problem in an alternating fashion. The performance of the proposed method is evaluated through the DrugBank dataset, and comparisons are provided against state-of-the-art techniques. The results demonstrate the superior performance of GRPMF when compared to its counterparts.

User-Centric Federated Matrix Factorization Based on Differential Privacy

Matrix factorization is a popular recommendation method used in many fields, but it often lacks sufficient privacy protection. To address privacy concerns in connected living, we propose a user-centric federated matrix factorization model based on differential privacy (FMF-DP). First, we design a federated matrix factorization framework where each user is a participant, and only uploads their gradients after completing the training task locally. Second, we introduce the user-average-rating bias into the matrix factorization method to improve accuracy. Finally, we use differential privacy to enhance privacy protection without compromising computational efficiency. We have implemented a prototype of FMF-DP and tested it on several datasets, showing promising results. Compared with the baseline model, the RMSE is reduced by 0.25 on average, and the computational efficiency is also improved.

SSNMDI: a novel joint learning model of semi-supervised non-negative matrix factorization and data imputation for clustering of single-cell RNA-seq data.

Single-cell RNA sequencing (scRNA-seq) technology attracts extensive attention in the biomedical field. It can be used to measure gene expression and analyze the transcriptome at the single-cell level, enabling the identification of cell types based on unsupervised clustering. Data imputation and dimension reduction are conducted before clustering because scRNA-seq has a high 'dropout' rate, noise and linear inseparability. However, independence of dimension reduction, imputation and clustering cannot fully characterize the pattern of the scRNA-seq data, resulting in poor clustering performance. Herein, we propose a novel and accurate algorithm, SSNMDI, that utilizes a joint learning approach to simultaneously perform imputation, dimensionality reduction and cell clustering in a non-negative matrix factorization (NMF) framework. In addition, we integrate the cell annotation as prior information, then transform the joint learning into a semi-supervised NMF model. Through experiments on 14 datasets, we demonstrate that SSNMDI has a faster convergence speed, better dimensionality reduction performance and a more accurate cell clustering performance than previous methods, providing an accurate and robust strategy for analyzing scRNA-seq data. Biological analysis are also conducted to validate the biological significance of our method, including pseudotime analysis, gene ontology and survival analysis. We believe that we are among the first to introduce imputation, partial label information, dimension reduction and clustering to the single-cell field. The source code for SSNMDI is available at https://github.com/yushanqiu/SSNMDI.