Matrix Factorization Framework Research Articles

A novel hyperspectral unmixing model based on multilayer NMF with Hoyer’s projection

Hyperspectral remote sensing is an important earth observation method with wide application. But the low spatial resolution of hyperspectral images makes it difficult to distinguish the ground objects. The hyperspectral image unmixing is a task to estimate the spectral signatures and corresponding fractional abundances. However, the unmixing speed and efficiency are still limited by traditional structures. In this paper, a novel multilayer nonnegative matrix factorization framework is proposed with Hoyer’s projector, called HP-MLNMF. The well-known framework, multilayer nonnegative factorization (MLNMF), is completely restructured and enhanced by introducing the Hoyer’s projector to provide the iteration directivity of the structure in the unmixing process. Besides, a novel sparse constraint to spectral signatures suitable for this structure is found as l1/4-norm based on some experimental discussions. Moreover, the lp-norm is utilized to find the possible sparest solution for abundance terms. Finally, HP-MLNMF is compared with some representative and state-of-art methods on synthetic and real-world hyperspectral image datasets. Experiments indicate that our method performances well in most cases.

Multi-modal Discrete Collaborative Filtering for Efficient Cold-start Recommendation

Hashing is an effective technique to improve the efficiency of large-scale recommender system by representing both users and items into binary codes. However, existing hashing-based recommendation methods still suffer from two important problems: 1) Cold-start. They employ the user-item interactions and single auxiliary information to learn the binary hash codes. But the full interaction history is not always available and the single auxiliary information may be missing. 2) Efficient optimization. They learn the hash codes with two-step relaxed optimization or one-step discrete hash optimization based on the cyclic coordinate descent, which results in significant quantization loss or still consumes considerable computation time. In this paper, we propose a Multi-modal Discrete Collaborative Filtering (MDCF) for efficient cold-start recommendation. We map the multi-modal features of users and items to a consensus Hamming space based on the matrix factorization framework. Specifically, a low-rank self-weighted multi-modal fusion module is designed to adaptively fuse the multi-modal features into binary hash codes. Additionally, to support large-scale recommendation, a fast discrete optimization method based on augmented Lagrangian multiplier is developed to directly compute the binary hash codes with simple operations. Experiments show the superior performance of the proposed method over state-of-the-art baselines.

FeatureMF: An Item Feature Enriched Matrix Factorization Model for Item Recommendation

Matrix Factorization (MF) is one of the most successful Collaborative Filtering (CF) techniques used in recommender systems due to its effectiveness and ability to deal with very large user-item rating matrix. However, when the rating matrix sparseness increases its performance deteriorates. Expanding MF to include side-information of users and items has been shown by many researchers both to improve general recommendation performance and to help alleviate the data-sparsity and cold-start issues in CF. In regard to item feature side-information, most schemes incorporate this information through a two stage process: intermediate results (e.g., on item similarity) are first computed based on item attributes; these are then combined with MF. In this paper, focussing on item side-information, we propose a model that directly incorporates item features into the MF framework in a single step process. The model, which we name FeatureMF, does this by projecting every available attribute datum in each of the item features into the same latent factor space with users and items, thereby in effect enriching item representation in MF. Results are presented of comparative performance experiments of the model against three state-of-the-art item information enriched models, as well as against four reference benchmark models, using two public real-world datasets, Douban and Yelp, with four training:test ratio scenarios each. It is shown to yield the best recommendation performance over all these models across all contexts including data-sparsity situations, in particular, achieving over 0.9% to over 6.5% MAE recommendation performance improvement over the next best model, HERec. FeatureMF is also found to alleviate cold start and to scale well, almost linearly, in regard to computational time, as a function of dataset size.

An Alternating Rank-k Nonnegative Least Squares Framework (ARkNLS) for Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is a prominent technique for data dimensionality reduction that has been widely used for text mining, computer vision, pattern discovery, and bioinformatics. In this paper, a framework called ARkNLS (Alternating Rank-k Nonnegativity constrained Least Squares) is proposed for computing NMF. First, a recursive formula for the solution of the rank-k nonnegativity-constrained least squares (NLS) is established. This recursive formula can be used to derive the closed-form solution for the Rank-k NLS problem for any integer k $\ge$ 1. As a result, each subproblem for an alternating rank-k nonnegative least squares framework can be obtained based on this closed form solution. Assuming that all matrices involved in rank-k NLS in the context of NMF computation are of full rank, two of the currently best NMF algorithms HALS (hierarchical alternating least squares) and ANLS-BPP (Alternating NLS based on Block Principal Pivoting) can be considered as special cases of ARkNLS with k = 1 and k = r for rank r NMF, respectively. This paper is then focused on the framework with k = 3, which leads to a new algorithm for NMF via the closed-form solution of the rank-3 NLS problem. Furthermore, a new strategy that efficiently overcomes the potential singularity problem in rank-3 NLS within the context of NMF computation is also presented. Extensive numerical comparisons using real and synthetic data sets demonstrate that the proposed algorithm provides state-of-the-art performance in terms of computational accuracy and cpu time.

Constrained Nonnegative Matrix Factorization for Blind Hyperspectral Unmixing Incorporating Endmember Independence

Hyperspectral unmixing (HU) has become an important technique in exploiting hyperspectral data since it decomposes a mixed pixel into a collection of endmembers weighted by fractional abundances. The endmembers of a hyperspectral image (HSI) are more likely to be generated by independent sources and be mixed in a macroscopic degree before arriving at the sensor element of the imaging spectrometer as mixed spectra. Over the past few decades, many attempts have focused on imposing auxiliary regularizes on the conventional nonnegative matrix factorization (NMF) framework in order to effectively unmix these mixed spectra. As a promising step toward finding an optimum regularizer to extract endmembers, this paper presents a novel blind HU algorithm, referred to as Kurtosis-based Smooth Nonnegative Matrix Factorization (KbSNMF) which incorporates a novel regularizer based on the statistical independence of the probability density functions of endmember spectra. Imposing this regularizer on the conventional NMF framework promotes the extraction of independent endmembers while further enhancing the parts-based representation of data. Experiments conducted on diverse synthetic HSI datasets (with numerous numbers of endmembers, spectral bands, pixels, and noise levels) and three standard real HSI datasets demonstrate the validity of the proposed KbSNMF algorithm compared to several stateof- the-art NMF-based HU baselines. The proposed algorithm exhibits superior performance especially in terms of extracting endmember spectra from hyperspectral data; therefore, it could uplift the performance of recent deep learning HU methods which utilize the endmember spectra as supervisory input data for abundance extraction.

Recovering Joint Probability of Discrete Random Variables From Pairwise Marginals

Learning the joint probability of random variables (RVs) is the cornerstone of statistical signal processing and machine learning. However, direct nonparametric estimation for high-dimensional joint probability is in general impossible, due to the curse of dimensionality. Recent work has proposed to recover the joint probability mass function (PMF) of an arbitrary number of RVs from three-dimensional marginals, leveraging the algebraic properties of low-rank tensor decomposition and the (unknown) dependence among the RVs. Nonetheless, accurately estimating three-dimensional marginals can still be costly in terms of sample complexity, affecting the performance of this line of work in practice in the sample-starved regime. Using three-dimensional marginals also involves challenging tensor decomposition problems whose tractability is unclear. This work puts forth a new framework for learning the joint PMF using only pairwise marginals, which naturally enjoys a lower sample complexity relative to the third-order ones. A coupled nonnegative matrix factorization (CNMF) framework is developed, and its joint PMF recovery guarantees under various conditions are analyzed. Our method also features a Gram--Schmidt (GS)-like algorithm that exhibits competitive runtime performance. The algorithm is shown to provably recover the joint PMF up to bounded error in finite iterations, under reasonable conditions. It is also shown that a recently proposed economical expectation maximization (EM) algorithm guarantees to improve upon the GS-like algorithm's output, thereby further lifting up the accuracy and efficiency. Real-data experiments are employed to showcase the effectiveness.

Matrix Factorization with Topic and Sentiment Analysis for Rating Prediction

In today's online services, users' feedback such as numerical rating, textual review, time of purchase, and so on for each item is often encouraged to provide. Managers of online services utilize the feedback to improve the quality of their services, or user experience. For example, many recommender systems predict the items that the users may like and purchase in the future using users' historical ratings. With the increase of user data in the systems, more detailed and interpretable information about item features and user sentiments can be extracted from textual reviews that are relative to ratings. In this paper, we propose a novel topic and sentiment matrix factorization model, which leverages both topic and sentiment drawn from the reviews simultaneously. First, we conduct topic analysis and sentiment analysis of reviews using Latent Dirichlet Allocation (LDA) and lexicon construction technique, respectively. Second, we combine the user consistency, which is calculated from his/her reviews and ratings, and helpful votes from other users of reviews to obtain a reliability measure to weight the ratings. Third, we integrate these three parts into the matrix factorization framework for the prediction of ratings. Our experimental comparison using Amazon datasets indicates that the proposed method significantly improves performance compared to traditional matrix factorization up to 14.12%.

Motion-Aware Structured Matrix Factorization for Foreground Detection in Complex Scenes

Foreground detection is one of the key steps in computer vision applications. Many foreground and background models have been proposed and achieved promising performance in static scenes. However, due to challenges such as dynamic background, irregular movement, and noise, most algorithms degrade sharply in complex scenes. To address the problem, we propose a motion-aware structured matrix factorization approach (MSMF), which integrates the structural and spatiotemporal motion information into a unified sparse-low-rank matrix factorization framework. Technologically, it has three main contributions: First, a variant of structured sparsity-inducing norm is proposed to constrain both structure and sparsity of foreground. The model is robust to the statistical variability of the underlying foreground pixels in complex scenes. Second, to capture the ambiguous pixels, a spatiotemporal cube-based motion trajectory is extracted for assisting matrix factorization. Finally, to solve the optimization problem of structured matrix factorization, we develop an augmented Lagrange multiplier method with the alternating direction strategy and Douglas-Rachford monotone operator splitting algorithm. Experiments demonstrate that the proposed approach achieves impressive performance in separating irregular moving foreground while suppressing the dynamic background and the noise, and outperforms some state-of-the-art algorithms.

Tracking Knowledge Structures and Proficiencies of Students With Learning Transfer

In online intelligent education systems, to offer proactive studying services to students (e.g., learning path recommendation), a crucial demand is to track students’ knowledge mastery levels over time. However, existing works ignore the impact of learning transfer on knowledge tracing and only track knowledge proficiency. Knowledge proficiency alone cannot fully reflect students’ knowledge mastery levels. A student’s knowledge structure (the similarities and differences within knowledge concepts) and abstract principle mastery level (common attributes among knowledge concepts, such as learning methods) also need to be tracked. To this end, we propose a novel multilevel Knowledge Tracing model with Learning Transfer (KTLT) to track students’ knowledge mastery levels. First, we clarify the relationships among abstract principles, knowledge structure, and knowledge proficiency by utilizing the learning transfer theory in educational psychology. Then, we associate each problem with a knowledge vector in which each element represents an explicit knowledge concept by leveraging educational priors (i.e., a Q-matrix). Correspondingly, each student is represented as a knowledge vector (knowledge proficiency) and an abstract principle vector (abstract principle mastery). Given a student’s knowledge and abstract principle vector over time, we use the learning and forgetting curve as priors to capture the student’s knowledge proficiency and abstract principle mastery level over time. Furthermore, we embed the knowledge concept by using a student’s abstract principle mastery level and obtain a personalized knowledge relevance matrix (a student’s knowledge structure) by calculating the cosine similarity among the knowledge embedding results. Finally, we design a probabilistic matrix factorization framework by combining student and problem priors for tracking a student’s knowledge mastery. Extensive experiments on two real-world datasets demonstrate both the effectiveness and explanatory ability of the KTLT.

CoGAPS 3: Bayesian non-negative matrix factorization for single-cell analysis with asynchronous updates and sparse data structures

BackgroundBayesian factorization methods, including Coordinated Gene Activity in Pattern Sets (CoGAPS), are emerging as powerful analysis tools for single cell data. However, these methods have greater computational costs than their gradient-based counterparts. These costs are often prohibitive for analysis of large single-cell datasets. Many such methods can be run in parallel which enables this limitation to be overcome by running on more powerful hardware. However, the constraints imposed by the prior distributions in CoGAPS limit the applicability of parallelization methods to enhance computational efficiency for single-cell analysis.ResultsWe developed a new software framework for parallel matrix factorization in Version 3 of the CoGAPS R/Bioconductor package to overcome the computational limitations of Bayesian matrix factorization for single cell data analysis. This parallelization framework provides asynchronous updates for sequential updating steps of the algorithm to enhance computational efficiency. These algorithmic advances were coupled with new software architecture and sparse data structures to reduce the memory overhead for single-cell data.ConclusionsAltogether our new software enhance the efficiency of the CoGAPS Bayesian matrix factorization algorithm so that it can analyze 1000 times more cells, enabling factorization of large single-cell data sets.

A kernel non-negative matrix factorization framework for single cell clustering

The emergence of single-cell RNA-sequencing is ideally placed to unravel cellular heterogeneity in biological systems, an extremely challenging problem in single cell RNA-sequencing studies. However, most current computational approaches lack the sensitivity to reliably detect nonlinear gene-gene relationships masked by dropout events. We proposed a kernel non-negative matrix factorization framework for detecting nonlinear relationships among genes, where the new kernel is developed using kernel tricks on cellular differentiability correlation. The newly constructed kernel not only provides a description on the gene-gene relationship, but also helps to build a new low-dimensional representation on the original data. Besides, we developed an efficient method for determining the optimal cluster number within each data set with the usage of Diffusion Maps. The proposed algorithm is further compared with representative algorithms: SC3 and several other state-of-the-art clustering methods, on several benchmark or real scRNA-Seq datasets using internal criteria (clustering number accuracy) and external criteria (Adjusted rand index and Normalized mutual information) to show effectiveness of our method.

Dimensionality reduction for single cell RNA sequencing data using constrained robust non-negative matrix factorization.

Single cell RNA-sequencing (scRNA-seq) technology, a powerful tool for analyzing the entire transcriptome at single cell level, is receiving increasing research attention. The presence of dropouts is an important characteristic of scRNA-seq data that may affect the performance of downstream analyses, such as dimensionality reduction and clustering. Cells sequenced to lower depths tend to have more dropouts than those sequenced to greater depths. In this study, we aimed to develop a dimensionality reduction method to address both dropouts and the non-negativity constraints in scRNA-seq data. The developed method simultaneously performs dimensionality reduction and dropout imputation under the non-negative matrix factorization (NMF) framework. The dropouts were modeled as a non-negative sparse matrix. Summation of the observed data matrix and dropout matrix was approximated by NMF. To ensure the sparsity pattern was maintained, a weighted ℓ1 penalty that took into account the dependency of dropouts on the sequencing depth in each cell was imposed. An efficient algorithm was developed to solve the proposed optimization problem. Experiments using both synthetic data and real data showed that dimensionality reduction via the proposed method afforded more robust clustering results compared with those obtained from the existing methods, and that dropout imputation improved the differential expression analysis.

TSCMF: Temporal and social collective matrix factorization model for recommender systems

In real-world recommender systems, user preferences are dynamic and typically change over time. Capturing the temporal dynamics of user preferences is essential to design an efficient personalized recommender system and has recently attracted significant attention. In this paper, we consider user preferences change individually over time. Moreover, based on the intuition that social influence can affect the users’ preferences in a recommender system, we propose a Temporal and Social Collective Matrix Factorization model called TSCMF for recommendation. We jointly factorize the users’ rating information and social trust information in a collective matrix factorization framework by introducing a joint objective function. We model user dynamics into this framework by learning a transition matrix of user preferences between two successive time periods for each individual user. We present an efficient optimization algorithm based on stochastic gradient descent for solving the objective function. The experiments on a real-world dataset illustrate that the proposed model outperforms the competitive methods. Moreover, the complexity analysis demonstrates that the proposed model can be scaled up to large datasets.

Analysis of Oral Carcinogenesis Factors Based on Non Negative Matrix Evolutionary Spectrum Clustering Decomposition

To explore the risk factors of Oral Submucosal fibrous carcinogenesis and provide theoretical basis for selfprevention and clinical treatment. First of all, an evolutionary non negative matrix factorization framework based on clustering quality is proposed for dynamic oral carcinogenesis detection. Then, the equivalence among evolutionary spectral clustering, ENMF and density optimization of evolutionary modules is proved theoretically. At the same time, based on this equivalence, a new semi-supervised ENMF algorithm is proposed by adding a priori information to ENMF without increasing the time complexity. Using this algorithm, 42 patients with Oral Submucosal Fibrosis and 40 non-cancerous patients with oral submucosal fibrosis were analyzed by single factor and multi-factor analysis to study the quantitative relationship between Oral Submucosal Fibrosis and 14 related factors. The results suggested that age, betel nut chewing duration, smoking duration and leukoplakia or lichen planus were the four factors which were introduced into the cluster quality analysis. Age, betel nut chewing duration, smoking duration and complications of leukoplakia or lichen planus are risk factors for Oral Submucosal fibrous carcinogenesis.

Incorporating side information into Robust Matrix Factorization with Bayesian Quantile Regression

Matrix Factorization is a widely used technique for modeling pairwise and matrix-like data. It is frequently used in pattern recognition, topic analysis and other areas. Side information is often available, however utilization of this additional information is problematic in the pure matrix factorization framework. This article proposes a novel method of utilizing side information by combining arbitrary nonlinear Quantile Regression model and Matrix Factorization under Bayesian framework. Gradient-free optimization procedure with the novel Surrogate Function is used to solve the resulting MAP estimator. The model performance has been evaluated on real data-sets.

Multi-source information fusion based heterogeneous network embedding

Heterogeneous network embedding aims to learn a mapping between network data in original topological space and vectored data in low dimensional latent space, while encoding valuable information, such as structural and semantic information. The resulting vector representation has shown promising performance for extensive real-world applications, such as node classification and node clustering. However, most of existing methods merely focus on modeling network structural information, ignoring the rich multi-source information of different types of nodes. In this paper, we propose a novel Multi-source Information Fusion based Heterogeneous Network Embedding (MIFHNE) approach. We first capture the semantic information using the strategy of meta-graph based random walk. Subsequently, we jointly model the structural proximity, attribute information and label information in the framework of Nonnegative Matrix Factorization (NMF). Theoretical proofs and comprehensive experiments on two real-world heterogeneous network datasets demonstrate the feasibility and effectiveness of our approach.

Visual Tactile Fusion Object Clustering

Object clustering, aiming at grouping similar objects into one cluster with an unsupervised strategy, has been extensively-studied among various data-driven applications. However, most existing state-of-the-art object clustering methods (e.g., single-view or multi-view clustering methods) only explore visual information, while ignoring one of most important sensing modalities, i.e., tactile information which can help capture different object properties and further boost the performance of object clustering task. To effectively benefit both visual and tactile modalities for object clustering, in this paper, we propose a deep Auto-Encoder-like Non-negative Matrix Factorization framework for visual-tactile fusion clustering. Specifically, deep matrix factorization constrained by an under-complete Auto-Encoder-like architecture is employed to jointly learn hierarchical expression of visual-tactile fusion data, and preserve the local structure of data generating distribution of visual and tactile modalities. Meanwhile, a graph regularizer is introduced to capture the intrinsic relations of data samples within each modality. Furthermore, we propose a modality-level consensus regularizer to effectively align the visual and tactile data in a common subspace in which the gap between visual and tactile data is mitigated. For the model optimization, we present an efficient alternating minimization strategy to solve our proposed model. Finally, we conduct extensive experiments on public datasets to verify the effectiveness of our framework.

Discrimination-Aware Projected Matrix Factorization

Non-negative Matrix Factorization (NMF) has been one of the most popular clustering techniques in machine leaning, and involves various real-world applications. Most existing works perform matrix factorization on high-dimensional data directly. However, the intrinsic data structure is always hidden within the low-dimensional subspace. And, the redundant features within the input space may affect the final result adversely. In this paper, a new unsupervised matrix factorization method, Discrimination-aware Projected Matrix Factorization (DPMF), is proposed for data clustering. The main contributions are threefold: (1) The linear discriminant analysis is jointly incorporated into the unsupervised matrix factorization framework, so the clustering can be accomplished in the discriminant subspace. (2) The manifold regularization is introduced to perceive the geometric information, and the ${\ell _{2,1}}$ l 2 , 1 -norm is utilized to improve the robustness. (3) An efficient optimization algorithm is designed to solve the proposed problem with proved convergence. Experimental results on one toy dataset and eight real-world benchmarks show the effectiveness of the proposed method.

A fusion collaborative filtering method for sparse data in recommender systems

Collaborative filtering is a fundamental technique in recommender systems, for which memory-based and matrix-factorization-based collaborative filtering are the two types of widely used methods. However, the performance of these two types of methods is limited in the case of sparse data, particularly with extremely sparse data. To improve the effectiveness of the methods in a sparse scenario, this paper proposes a multi-factor similarity measure that captures linear and nonlinear correlations between users resulting from extreme behavior. Subsequently, a fusion method that simultaneously considers the multi-factor similarity and the global rating information in a probability matrix factorization framework is proposed. In our framework, users’ local relations are integrated into the global ratings optimization process, so that prediction accuracy and robustness are improved in sparse data, particularly in extremely sparse data. To verify the performance of the proposed methods, we conduct experiments on four public datasets. The experimental results show that the fusion method is superior to the typical matrix factorization models used in collaborative filtering and significantly improves both the prediction results and robustness in sparse data.

CAPER: Context-Aware Personalized Emoji Recommendation

With the popularity of social platforms, emoji appears and becomes extremely popular with a large number of users. It expresses more beyond plaintexts and makes the content more vivid. Using appropriate emojis in messages and microblog posts makes you lovely and friendly. Recently, emoji recommendation becomes a significant task since it is hard to choose the appropriate one from thousands of emoji candidates. In this paper, we propose a Context-Aware Personalized Emoji Recommendation (CAPER) model fusing the contextual information and the personal information. It is to learn latent factors of contextual and personal information through a score-ranking matrix factorization framework. The personal factors such as user preference, user gender, and the current time can make the recommended emojis meet users' individual needs. Moreover, we consider the co-occurrence factors of the emojis which could improve the recommendation accuracy. We conduct a series of experiments on the real-world datasets, and experiment results show better performance of our model than existing methods, demonstrating the effectiveness of the considering contextual and personal factors.