Discrete Matrix Factorization Research Articles

Rank-r Discrete Matrix Factorization for Anchor Graph Clustering

Considering many graph clustering methods are with quadratic or cubic time complexity and need post-processing to obtain the discrete solution. Combining with the anchor graph, we study a novel graph clustering model called Rank-<inline-formula><tex-math notation="LaTeX">$r$</tex-math></inline-formula> Discrete Matrix Factorization (DMF-RR), which is linear time complexity, and motivated by nonnegative matrix factorization (NMF). Instead of constraining the factor matrices of NMF to be nonnegative as many existed methods, we constrain them to indicator matrices. Thus, DMF-RR can obtain the discrete solution by directly solving the original problem without post-processing. Furthermore, considering the greater similarity between samples of the same category, an anchor graph is constructed as an input to capture essential clustering structure by utilizing the duality information between samples and anchors. Subsequently, an efficient and simple algorithm is proposed due to the nature of indicator matrices. Extensive experiments performed on synthetic and real-world datasets demonstrate the superiority of DMF-RR.

EDMH: Efficient discrete matrix factorization hashing for multi-modal similarity retrieval

Hashing has been an emerging topic and has recently attracted widespread attention in multi-modal similarity search applications. However, most existing approaches rely on relaxation schemes to generate binary codes, leading to large quantization errors. In addition, amounts of existing approaches embed labels into the pairwise similarity matrix, leading to expensive time and space costs and losing category information. To address these issues, we propose an Efficient Discrete Matrix factorization Hashing (EDMH). Specifically, EDMH first learns the latent subspaces for individual modality through matrix factorization strategy, which preserves the semantic structure representation information of each modality. In particular, we develop a semantic label offset embedding learning strategy, improving the stability of label embedding regression. Furthermore, we design an efficient discrete optimization scheme to generate compact binary codes discretely. Eventually, we present two efficient learning strategies EDMH-L and EDMH-S to pursue high-quality hash functions. Extensive experiments on various widely-used databases verify that the proposed algorithms produce significant performance and outperform some state-of-the-art approaches, with an average improvement of 2.50% (for Wiki), 2.66% (for MIRFlickr) and 2.25% (for NUS-WIDE) over the best available results, respectively.

Discrete matrix factorization cross-modal hashing with multi-similarity consistency

AbstractRecently, matrix factorization-based hashing has gained wide attention because of its strong subspace learning ability and high search efficiency. However, some problems need to be further addressed. First, uniform hash codes can be generated by collective matrix factorization, but they often cause serious loss, degrading the quality of hash codes. Second, most of them preserve the absolute similarity simply in hash codes, failing to capture the inherent semantic affinity among training data. To overcome these obstacles, we propose a Discrete Multi-similarity Consistent Matrix Factorization Hashing (DMCMFH). Specifically, an individual subspace is first learned by matrix factorization and multi-similarity consistency for each modality. Then, the subspaces are aligned by a shared semantic space to generate homogenous hash codes. Finally, an iterative-based discrete optimization scheme is presented to reduce the quantization loss. We conduct quantitative experiments on three datasets, MSCOCO, Mirflickr25K and NUS-WIDE. Compared with supervised baseline methods, DMCMFH achieves increases of $$0.22\%$$ 0.22 % , $$3.00\%$$ 3.00 % and $$0.79\%$$ 0.79 % on the image-query-text tasks for three datasets respectively, and achieves increases of $$0.21\%$$ 0.21 % , $$1.62\%$$ 1.62 % and $$0.50\%$$ 0.50 % on the text-query-image tasks for three datasets respectively.

Scalable Discrete Matrix Factorization and Semantic Autoencoder for Cross-Media Retrieval.

Hashing methods have sparked great attention on multimedia tasks due to their effectiveness and efficiency. However, most existing methods generate binary codes by relaxing the binary constraints, which may cause large quantization error. In addition, most supervised cross-modal approaches preserve the similarity relationship by constructing an n×n large-size similarity matrix, which requires huge computation, making these methods unscalable. To address the above challenges, this article presents a novel algorithm, called scalable discrete matrix factorization and semantic autoencoder method (SDMSA). SDMSA is a two-stage method. In the first stage, the matrix factorization scheme is utilized to learn the latent semantic information, the label matrix is incorporated into the loss function instead of the similarity matrix. Thereafter, the binary codes can be generated by the latent representations. During optimization, we can avoid manipulating a large n×n similarity matrix, and the hash codes can be generated directly. In the second stage, a novel hash function learning scheme based on the autoencoder is proposed. The encoder-decoder paradigm aims to learn projections, the feature vectors are projected to code vectors by encoder, and the code vectors are projected back to the original feature vectors by the decoder. The encoder-decoder scheme ensures the embedding can well preserve both the semantic and feature information. Specifically, two algorithms SDMSA-lin and SDMSA-ker are developed under the SDMSA framework. Owing to the merit of SDMSA, we can get more semantically meaningful binary hash codes. Extensive experiments on several databases show that SDMSA-lin and SDMSA-ker achieve promising performance.

Robust and discrete matrix factorization hashing for cross-modal retrieval

Hashing based methods have gained great success for cross-modal similarity search, due to its fast query speed and low storage cost. However, there are some challenging problems that need to be further solved: 1) Many approaches are sensitive to noises and outliers, because ℓ2 norm is utilized in the objective function, the error may be amplified. 2) Most existing methods take relaxation or rounding scheme to generate binary codes, causing a large quantization loss. 3) Many supervised cross-media algorithms usually take a large n×n matrix to preserve the similarity relationship, leading to large calculation and making them unscalable. To mitigate these challenges, we develop a novel cross-media search algorithm, i.e., robust and discrete matrix factorization hashing, dubbed RDMH. The method takes a two-step strategy. In the first phase, the ℓ2,1 norm is utilized to improve the robustness, which makes our model not sensitive to noises and outliers. We can learn the hash codes directly by the proposed discrete optimization method instead of relaxation scheme, avoiding the large quantization loss. Moreover, RDMH correlates the hash codes and semantic labels directly instead of manipulating the large similarity matrix. In the second phase, we propose an autoencoder strategy to learn the hash functions, more valuable information can be preserved and making the hash functions more powerful. Comprehensive experiments on several databases demonstrate the superior performance and efficacy of the developed RDMH.

Discrete matrix factorization hashing for cross-modal retrieval

Discrete matrix factorization hashing for cross-modal retrieval

Semi-discrete Matrix Factorization

Discrete matrix factorization (DMF) has been a promising solution to improve the inferring efficiency of matrix factorization (MF) against the rapidly growing numbers of users and items. However, DMF suffers from a serious encoding loss due to its oversimplified modeling on the original data geometry. In this article, we propose a semi-discrete matrix factorization (SDMF) model to combine the predicting efficacy of MF with the inferring efficiency of DMF. It first learns real-valued latent features by MF, and then, taking them as group-wise and point-wise smoothness, learns binary codes in the DMF framework, for preserving the geometrical structures collectively hidden in users and items, as well as aligning binary codes originated from Hamming space with their real-valued counterparts learned from vector space. Particularly, we devise a computationally efficient optimization algorithm to estimate model parameters. Extensive evaluations on three real-world datasets clearly demonstrate the superiority of our SDMF model over state-of-the-art hash-based recommendation methods.

Cross-view hashing via supervised deep discrete matrix factorization

Matrix factorization has been utilized for the task of cross-view hashing, where basis functions are learned to map data from different views to the same hamming embedding. It is possible that the basis functions between the hamming embedding and the original data matrix contain rather complex hierarchical information, which existing work can not capture. In addition, previous work employs relaxation technique in the matrix factorization based hashing which may lead to large quantization error. To address these issues, this paper presents a novel Supervised Discrete Deep Matrix Factorization (SDDMF) for cross-view hashing. We introduce deep matrix factorization so that SDDMF is able to learn a set of hierarchical basis functions and unified binary codes from different views. In addition, a classification error term is incorporated into the objective to learn discriminative binary codes. We then employ a linearization technique to directly optimize the discrete constraints which can significantly reduce the quantization error. Experimental results on three standard datasets with image-text modalities verify that SDDMF significantly outperforms several state-of-the-art methods.

Semantic-enhanced discrete matrix factorization hashing for heterogeneous modal matching

The simultaneous hash representation of heterogeneous modalities has shown its excellent performance in the multi-modal learning community. However, the binary discrete constraint of hash brings a great challenge to the common hash representation of heterogeneous modal data. To resolve the issue, this paper proposes a novel discrete hash method, referred to as Semantic enhanced Discrete Matrix Factorization Hashing (SDMFH). SDMFH directly extracts the common discrete hash representation of all modalities from the reconstructed semantic intermodal similarity graph, which makes the hash codes more discriminative. Meanwhile, the semantic labels are regressed to the extracted discrete hash book, so as to further strengthen the discriminating ability of the learned discrete hash book. Moreover, a linear embedding from the kernel space of the original data to the Hamming space is explored to generate the common hash codes of each modality to ensure the consistency of the hash codes of heterogeneous modalities. More importantly, we develop an efficient discrete iterative algorithm based on Stiefel manifold that can directly learn the discrete hash book in a closed form, thus both avoiding quantization loss caused by discrete relaxation and reducing computational complexity. Experimental results on three benchmark data sets show that SDMFH performs better than several state-of-the-art methods for heterogeneous modal matching tasks.

Bidirectional Discrete Matrix Factorization Hashing for Image Search.

Unsupervised image hashing has recently gained significant momentum due to the scarcity of reliable supervision knowledge, such as class labels and pairwise relationship. Previous unsupervised methods heavily rely on constructing sufficiently large affinity matrix for exploring the geometric structure of data. Nevertheless, due to lack of adequately preserving the intrinsic information of original visual data, satisfactory performance can hardly be achieved. In this article, we propose a novel approach, called bidirectional discrete matrix factorization hashing (BDMFH), which alternates two mutually promoted processes of 1) learning binary codes from data and 2) recovering data from the binary codes. In particular, we design the inverse factorization model, which enforces the learned binary codes inheriting intrinsic structure from the original visual data. Moreover, we develop an efficient discrete optimization algorithm for the proposed BDMFH. Comprehensive experimental results on three large-scale benchmark datasets show that the proposed BDMFH not only significantly outperforms the state-of-the-arts but also provides the satisfactory computational efficiency.

SCRATCH: A Scalable Discrete Matrix Factorization Hashing Framework for Cross-Modal Retrieval

In this paper, we present a novel supervised cross-modal hashing framework, namely Scalable disCRete mATrix faCtorization Hashing (SCRATCH). First, it utilizes collective matrix factorization on original features together with label semantic embedding, to learn the latent representations in a shared latent space. Thereafter, it generates binary hash codes based on the latent representations. During optimization, it avoids using a large $n\times n$ similarity matrix and generates hash codes discretely. Besides, based on different objective functions, learning strategy, and features, we further present three models in this framework, i.e., SCRATCH-o, SCRATCH-t, and SCRATCH-d. The first one is a one-step method, learning the hash functions and the binary codes in the same optimization problem. The second is a two-step method, which first generates the binary codes and then learns the hash functions based on the learned hash codes. The third one is a deep version of SCRATCH-t, which utilizes deep neural networks as hash functions. The extensive experiments on two widely used benchmark datasets demonstrate that SCRATCH-o and SCRATCH-t outperform some state-of-the-art shallow hashing methods for cross-modal retrieval. The SCRATCH-d also outperforms some state-of-the-art deep hashing models.

Discrete Matrix Factorization and Extension for Fast Item Recommendation

Binary representation of users and items can dramatically improve efficiency of recommendation and reduce size of recommendation models. However, learning optimal binary codes for them is challenging due to binary constraints, even if squared loss is optimized. In this article, we propose a general framework for discrete matrix factorization based on discrete optimization, which can 1) optimize multiple loss functions; 2) handle both explicit and implicit feedback datasets; and 3) take auxiliary information into account without any hyperparameters. To tackle the challenging discrete optimization problem, we propose block coordinate descent based on semidefinite relaxation of binary quadratic programming. We theoretically show that it is equivalent to discrete coordinate descent when only one coordinate is in each block. We extensively evaluate the proposed algorithms on eight real-world datasets. The results of evaluation show that they outperform the state-of-the-art baselines significantly and that auxiliary information of items improves recommendation performance. For better showing the advantages of binary representation, we further propose a two-stage recommender system, consisting of an item-recalling stage and a subsequent fine-ranking stage. Its extensive evaluation shows hashing can dramatically accelerate item recommendation with little degradation of accuracy.